[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:37 dist-test-slave-bzzj krb5kdc[6167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:37 dist-test-slave-bzzj krb5kdc[6167](info): set up 2 sockets
Sep 27 18:37:37 dist-test-slave-bzzj krb5kdc[6167](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 18:37:38.507723 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:38 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:38.633284 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:37823
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:37823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:38.736428 6180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:38.736651 6180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:38.736687 6180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:38.740543 6180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:38.740589 6180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:38.740607 6180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:38.740621 6180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:38.740638 6180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:38.745656 6180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:37823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:37823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:38.746537 6180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:38.747454 6180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:38.755337 6186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:38.755342 6185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:38.755388 6180 server_base.cc:1034] running on GCE node
W20260927 18:37:38.755347 6188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:38.756026 6180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:38.757030 6180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:38.758199 6180 hybrid_clock.cc:648] HybridClock initialized: now 1790534258758184 us; error 25 us; skew 500 ppm
Sep 27 18:37:38 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534258, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:38.760701 6180 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:38.761490 6180 webserver.cc:460] Webserver started at http://127.6.4.62:41573/ using document root <none> and password file <none>
I20260927 18:37:38.762001 6180 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:38.762061 6180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:38.762238 6180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:38.764083 6180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "afcd1aa6418a4dd589e1b03c9fa7c423"
format_stamp: "Formatted at 2026-09-27 18:37:38 on dist-test-slave-bzzj"
server_key: "11593788b95df26c74d3d48c29066df4"
server_key_iv: "cce517091ebb06b2db5a16c5ef8436a4"
server_key_version: "encryptionkey@0"
I20260927 18:37:38.764638 6180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "afcd1aa6418a4dd589e1b03c9fa7c423"
format_stamp: "Formatted at 2026-09-27 18:37:38 on dist-test-slave-bzzj"
server_key: "11593788b95df26c74d3d48c29066df4"
server_key_iv: "cce517091ebb06b2db5a16c5ef8436a4"
server_key_version: "encryptionkey@0"
I20260927 18:37:38.768041 6180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260927 18:37:38.770344 6195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:38.771528 6180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 18:37:38.771634 6180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "afcd1aa6418a4dd589e1b03c9fa7c423"
format_stamp: "Formatted at 2026-09-27 18:37:38 on dist-test-slave-bzzj"
server_key: "11593788b95df26c74d3d48c29066df4"
server_key_iv: "cce517091ebb06b2db5a16c5ef8436a4"
server_key_version: "encryptionkey@0"
I20260927 18:37:38.771732 6180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:38.781539 6180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:38.783987 6180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:38.784163 6180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:38.790414 6180 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:37823
I20260927 18:37:38.790431 6247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:37823 every 8 connection(s)
I20260927 18:37:38.791236 6180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:38.793841 6248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:38.798223 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6180
I20260927 18:37:38.798385 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:38.799278 6248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423: Bootstrap starting.
I20260927 18:37:38.799933 6160 external_mini_cluster.cc:1428] Setting key 3b731da29377d8465ef9fea6032c47de
I20260927 18:37:38.802222 6248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:38.803009 6248 log.cc:824] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:38.804790 6248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423: No bootstrap required, opened a new log
I20260927 18:37:38.808715 6248 raft_consensus.cc:348] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37823 } }
I20260927 18:37:38.808960 6248 raft_consensus.cc:374] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:38.809015 6248 raft_consensus.cc:729] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcd1aa6418a4dd589e1b03c9fa7c423, State: Initialized, Role: FOLLOWER
I20260927 18:37:38.809453 6248 consensus_queue.cc:260] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37823 } }
I20260927 18:37:38.809581 6248 raft_consensus.cc:388] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:38.809640 6248 raft_consensus.cc:482] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:38.809729 6248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:38.810485 6248 raft_consensus.cc:504] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37823 } }
I20260927 18:37:38.810781 6248 leader_election.cc:302] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afcd1aa6418a4dd589e1b03c9fa7c423; no voters:
I20260927 18:37:38.811038 6248 leader_election.cc:288] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:38.811158 6253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:38.811352 6253 raft_consensus.cc:686] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [term 1 LEADER]: Becoming Leader. State: Replica: afcd1aa6418a4dd589e1b03c9fa7c423, State: Running, Role: LEADER
I20260927 18:37:38.811632 6253 consensus_queue.cc:237] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37823 } }
I20260927 18:37:38.812229 6248 sys_catalog.cc:564] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:38.813293 6255 sys_catalog.cc:455] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afcd1aa6418a4dd589e1b03c9fa7c423. Latest consensus state: current_term: 1 leader_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37823 } } }
I20260927 18:37:38.813293 6254 sys_catalog.cc:455] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd1aa6418a4dd589e1b03c9fa7c423" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37823 } } }
I20260927 18:37:38.813442 6255 sys_catalog.cc:458] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:38.813442 6254 sys_catalog.cc:458] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:38.813827 6262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:38.816812 6262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 18:37:38 dist-test-slave-bzzj krb5kdc[6167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:38.822582 6262 catalog_manager.cc:1347] Generated new cluster ID: ba5fb397c3e44740acf3dd314ae836af
I20260927 18:37:38.822636 6262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:38.824419 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:38.803111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59154 (local address 127.6.4.62:37823)
0927 18:37:38.803520 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:38.803538 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:38.804699 (+ 1161us) server_negotiation.cc:407] Connection header received
0927 18:37:38.808285 (+ 3586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:38.808307 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:38.808684 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:38.809031 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:38.811086 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:38.812493 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:38.814190 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:38.814892 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:38.816265 (+ 1373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:38.819005 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:38.819012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:38.819023 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:38.819049 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:38.820611 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:38.821202 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:38.821207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:38.821211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:38.821322 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:38.821647 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:38.821649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:38.821650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:38.821999 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:38.822197 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:38.822946 (+ 749us) server_negotiation.cc:299] Negotiation successful
0927 18:37:38.823530 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"spinlock_wait_cycles":18688,"thread_start_us":132,"threads_started":1}
I20260927 18:37:38.835115 6262 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:38.835963 6262 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:38.842551 6262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afcd1aa6418a4dd589e1b03c9fa7c423: Generated new TSK 0
I20260927 18:37:38.843091 6262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:38.871372 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37823
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:38.973784 6274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:38.974010 6274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:38.974066 6274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:38.978070 6274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:38.978119 6274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:38.978199 6274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:38.983129 6274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:38.984035 6274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:38.984961 6274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:38.993429 6280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:38.993467 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:38.993566 6274 server_base.cc:1034] running on GCE node
W20260927 18:37:38.993439 6282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:38.994024 6274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:38.994586 6274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:38.995746 6274 hybrid_clock.cc:648] HybridClock initialized: now 1790534258995722 us; error 35 us; skew 500 ppm
Sep 27 18:37:38 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534258, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:38.998194 6274 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:38.998929 6274 webserver.cc:460] Webserver started at http://127.6.4.1:43553/ using document root <none> and password file <none>
I20260927 18:37:38.999419 6274 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:38.999480 6274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:38.999652 6274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:39.001542 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "6c1a91dfbf2d4624a92fba8194df289c"
format_stamp: "Formatted at 2026-09-27 18:37:38 on dist-test-slave-bzzj"
server_key: "f2f0f440f4d9a7fa1ae7028de954ae90"
server_key_iv: "e9dcbb25538657b686215877a222bc98"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.002053 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "6c1a91dfbf2d4624a92fba8194df289c"
format_stamp: "Formatted at 2026-09-27 18:37:38 on dist-test-slave-bzzj"
server_key: "f2f0f440f4d9a7fa1ae7028de954ae90"
server_key_iv: "e9dcbb25538657b686215877a222bc98"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.005373 6274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:37:39.007500 6289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.008632 6274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:39.008730 6274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "6c1a91dfbf2d4624a92fba8194df289c"
format_stamp: "Formatted at 2026-09-27 18:37:38 on dist-test-slave-bzzj"
server_key: "f2f0f440f4d9a7fa1ae7028de954ae90"
server_key_iv: "e9dcbb25538657b686215877a222bc98"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.008816 6274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:39.048208 6274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:39.050750 6274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:39.050908 6274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:39.051441 6274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:39.052233 6274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:39.052279 6274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.052331 6274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:39.052359 6274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.060326 6274 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:42551
I20260927 18:37:39.060354 6402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:42551 every 8 connection(s)
I20260927 18:37:39.061136 6274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:37:39.066745 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6274
I20260927 18:37:39.066845 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:39.067085 6160 external_mini_cluster.cc:1428] Setting key d8dade6adef38dd030cd28a7c37e84ba
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534258, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:39.074218 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.062801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:42291 (local address 127.6.4.62:37823)
0927 18:37:39.062937 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:39.062941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:39.063682 (+ 741us) server_negotiation.cc:407] Connection header received
0927 18:37:39.064984 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:39.064987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:39.065034 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:39.065103 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:39.066134 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.066770 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.068051 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.068360 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.068440 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.070881 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:39.070884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:39.070886 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:39.070907 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:39.072301 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.072798 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.072802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.072804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.072876 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.073191 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.073194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.073196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.073361 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:39.073444 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:39.073931 (+ 487us) server_negotiation.cc:299] Negotiation successful
0927 18:37:39.074060 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:37:39.074971 6405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.063130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37823 (local address 127.6.4.1:42291)
0927 18:37:39.063535 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:39.063575 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:39.064799 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:39.065254 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:39.065261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:39.065579 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:39.066025 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:39.066038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.066906 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.066910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:39.067938 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:39.067948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.068455 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.068516 (+ 61us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:39.068666 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.069179 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:39.069194 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:39.070710 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:39.072412 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:39.072417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:39.072431 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:39.072695 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:39.072960 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:39.072963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:39.072965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:39.073099 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:39.073468 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:39.073475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:39.073707 (+ 232us) client_negotiation.cc:769] Sending connection context
0927 18:37:39.073862 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 18:37:39.074160 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:37:39.076265 6403 heartbeater.cc:347] Connected to a master server at 127.6.4.62:37823
I20260927 18:37:39.076448 6403 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:39.076838 6403 heartbeater.cc:511] Master 127.6.4.62:37823 requested a full tablet report, sending...
I20260927 18:37:39.078269 6212 ts_manager.cc:194] Registered new tserver with Master: 6c1a91dfbf2d4624a92fba8194df289c (127.6.4.1:42551)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:37:39.079396 6212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:42291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:39.091233 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37823
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:39.199501 6408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:39.199718 6408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:39.199759 6408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:39.203522 6408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:39.203567 6408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:39.203621 6408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:39.208532 6408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:39.209407 6408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:39.210292 6408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:39.219805 6414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:39.219818 6416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:39.219838 6413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:39.220049 6408 server_base.cc:1034] running on GCE node
I20260927 18:37:39.220414 6408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:39.220981 6408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:39.222146 6408 hybrid_clock.cc:648] HybridClock initialized: now 1790534259222124 us; error 38 us; skew 500 ppm
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:39.224612 6408 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:39.225366 6408 webserver.cc:460] Webserver started at http://127.6.4.2:36893/ using document root <none> and password file <none>
I20260927 18:37:39.225858 6408 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:39.225919 6408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:39.226104 6408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:39.228034 6408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "2214b36c9ce24adcb51372cd85da2244"
format_stamp: "Formatted at 2026-09-27 18:37:39 on dist-test-slave-bzzj"
server_key: "66c47c6459999744ea34df4854a88b40"
server_key_iv: "97d408d1f6d8f0c1d091adb97a2e2abc"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.228575 6408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "2214b36c9ce24adcb51372cd85da2244"
format_stamp: "Formatted at 2026-09-27 18:37:39 on dist-test-slave-bzzj"
server_key: "66c47c6459999744ea34df4854a88b40"
server_key_iv: "97d408d1f6d8f0c1d091adb97a2e2abc"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.231906 6408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:39.234117 6423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.235289 6408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:39.235399 6408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "2214b36c9ce24adcb51372cd85da2244"
format_stamp: "Formatted at 2026-09-27 18:37:39 on dist-test-slave-bzzj"
server_key: "66c47c6459999744ea34df4854a88b40"
server_key_iv: "97d408d1f6d8f0c1d091adb97a2e2abc"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.235486 6408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:39.244045 6408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:39.246637 6408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:39.246797 6408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:39.247284 6408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:39.248028 6408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:39.248071 6408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.248175 6408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:39.248207 6408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.256243 6408 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:43085
I20260927 18:37:39.256297 6536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:43085 every 8 connection(s)
I20260927 18:37:39.257027 6408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:39.257937 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6408
I20260927 18:37:39.258047 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:39.258294 6160 external_mini_cluster.cc:1428] Setting key 4cee564e73b3bd6ec01ef5627e82a16a
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:37:39.270597 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.258681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:56105 (local address 127.6.4.62:37823)
0927 18:37:39.258804 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:39.258808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:39.260161 (+ 1353us) server_negotiation.cc:407] Connection header received
0927 18:37:39.261450 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:39.261454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:39.261514 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:39.261599 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:39.262657 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.263458 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.264602 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.264932 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.265041 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.267325 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:39.267329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:39.267332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:39.267358 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:39.268595 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.269078 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.269081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.269082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.269154 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.269494 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.269496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.269497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.269683 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:39.269752 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:39.270312 (+ 560us) server_negotiation.cc:299] Negotiation successful
0927 18:37:39.270437 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:37:39.271229 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.259071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37823 (local address 127.6.4.2:56105)
0927 18:37:39.260000 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:39.260051 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:39.261256 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:39.261744 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:39.261750 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:39.262067 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:39.262513 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:39.262526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.263578 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.263581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:39.264489 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:39.264498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.265019 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.265055 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:39.265196 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.265713 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:39.265729 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:39.267171 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:39.268702 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:39.268707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:39.268717 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:39.268982 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:39.269241 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:39.269245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:39.269247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:39.269414 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:39.269768 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:39.269773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:39.270060 (+ 287us) client_negotiation.cc:769] Sending connection context
0927 18:37:39.270225 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 18:37:39.270449 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":105,"threads_started":1}
I20260927 18:37:39.272300 6537 heartbeater.cc:347] Connected to a master server at 127.6.4.62:37823
I20260927 18:37:39.272500 6537 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:39.272922 6537 heartbeater.cc:511] Master 127.6.4.62:37823 requested a full tablet report, sending...
I20260927 18:37:39.273903 6212 ts_manager.cc:194] Registered new tserver with Master: 2214b36c9ce24adcb51372cd85da2244 (127.6.4.2:43085)
I20260927 18:37:39.274430 6212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:56105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:39.283835 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37823
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:39.389849 6542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:39.390085 6542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:39.390125 6542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:39.394088 6542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:39.394142 6542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:39.394202 6542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:39.399217 6542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:39.400198 6542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:39.401160 6542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:39.410720 6547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:39.410720 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:39.410828 6548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:39.410779 6542 server_base.cc:1034] running on GCE node
I20260927 18:37:39.411270 6542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:39.411864 6542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:39.413028 6542 hybrid_clock.cc:648] HybridClock initialized: now 1790534259412998 us; error 45 us; skew 500 ppm
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:39.415738 6542 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:39.416594 6542 webserver.cc:460] Webserver started at http://127.6.4.3:35161/ using document root <none> and password file <none>
I20260927 18:37:39.417117 6542 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:39.417174 6542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:39.417361 6542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:39.419292 6542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "c74204cb6c8f4d25bcf25e9a1cbec7b5"
format_stamp: "Formatted at 2026-09-27 18:37:39 on dist-test-slave-bzzj"
server_key: "7e0d04efa05855ddad66bd05240dfc5d"
server_key_iv: "ba9436356cedbb97a6ddbe1bd05f95f0"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.419813 6542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "c74204cb6c8f4d25bcf25e9a1cbec7b5"
format_stamp: "Formatted at 2026-09-27 18:37:39 on dist-test-slave-bzzj"
server_key: "7e0d04efa05855ddad66bd05240dfc5d"
server_key_iv: "ba9436356cedbb97a6ddbe1bd05f95f0"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.423316 6542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:37:39.425580 6557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.426672 6542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:39.426762 6542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "c74204cb6c8f4d25bcf25e9a1cbec7b5"
format_stamp: "Formatted at 2026-09-27 18:37:39 on dist-test-slave-bzzj"
server_key: "7e0d04efa05855ddad66bd05240dfc5d"
server_key_iv: "ba9436356cedbb97a6ddbe1bd05f95f0"
server_key_version: "encryptionkey@0"
I20260927 18:37:39.426859 6542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:39.449013 6542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:39.451896 6542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:39.452049 6542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:39.452584 6542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:39.453320 6542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:39.453366 6542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.453404 6542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:39.453424 6542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:39.461769 6542 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:46071
I20260927 18:37:39.461803 6670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:46071 every 8 connection(s)
I20260927 18:37:39.462574 6542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:37:39.469306 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6542
I20260927 18:37:39.469413 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:39.469664 6160 external_mini_cluster.cc:1428] Setting key 54272ec58a727ff7874c972f0e27d677
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:39.474821 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.464301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:41257 (local address 127.6.4.62:37823)
0927 18:37:39.464466 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:39.464470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:39.465244 (+ 774us) server_negotiation.cc:407] Connection header received
0927 18:37:39.466461 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:39.466464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:39.466522 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:39.466600 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:39.467621 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.468201 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.469315 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.469631 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.469737 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.472042 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:39.472044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:39.472046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:39.472069 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:39.473165 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.473607 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.473609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.473610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.473678 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.473996 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.473997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.473998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.474142 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:39.474221 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:39.474622 (+ 401us) server_negotiation.cc:299] Negotiation successful
0927 18:37:39.474712 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 18:37:39.475692 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.464684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37823 (local address 127.6.4.3:41257)
0927 18:37:39.465100 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:39.465140 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:39.466275 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:39.466754 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:39.466760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:39.467074 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:39.467509 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:39.467521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.468307 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.468310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:39.469191 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:39.469200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.469727 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.469779 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:39.469923 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.470426 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:39.470441 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:39.471902 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:39.473260 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:39.473265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:39.473275 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:39.473525 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:39.473779 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:39.473782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:39.473784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:39.473920 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:39.474227 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:39.474231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:39.474436 (+ 205us) client_negotiation.cc:769] Sending connection context
0927 18:37:39.474581 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 18:37:39.474811 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260927 18:37:39.476688 6671 heartbeater.cc:347] Connected to a master server at 127.6.4.62:37823
I20260927 18:37:39.476899 6671 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:39.477330 6671 heartbeater.cc:511] Master 127.6.4.62:37823 requested a full tablet report, sending...
I20260927 18:37:39.478248 6212 ts_manager.cc:194] Registered new tserver with Master: c74204cb6c8f4d25bcf25e9a1cbec7b5 (127.6.4.3:46071)
I20260927 18:37:39.478763 6212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:41257
I20260927 18:37:39.483733 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:39.507210 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.498697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.6.4.1:42551)
0927 18:37:39.498984 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:39.498995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:39.499018 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:37:39.499077 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:39.499081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:39.499254 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:39.499363 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:39.499934 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.500780 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.501569 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.501932 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.502017 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.504187 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:39.504196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:39.504209 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:39.504232 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:39.505537 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.505926 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.505932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.505933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.506009 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.506254 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.506257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.506259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.506497 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:39.506683 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:39.506899 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 18:37:39.507043 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
W20260927 18:37:39.508234 6317 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51066
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:39.532449 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:39.525018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.6.4.1:42551)
0927 18:37:39.525199 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:39.525203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:39.525271 (+ 68us) server_negotiation.cc:407] Connection header received
0927 18:37:39.525376 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:39.525378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:39.525419 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:39.525485 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:39.526056 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.526560 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.527302 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:39.527524 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:39.527598 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:39.529777 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:39.529780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:39.529782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:39.529803 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:39.530983 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.531367 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.531370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.531371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.531437 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:39.531682 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:39.531685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:39.531687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:39.531836 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:39.531943 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:39.532199 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 18:37:39.532302 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 18:37:39.533505 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6274
I20260927 18:37:39.539052 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6408
I20260927 18:37:39.544773 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6542
I20260927 18:37:39.549288 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6180
2026-09-27T18:37:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2002 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6685](info): set up 2 sockets
Sep 27 18:37:39 dist-test-slave-bzzj krb5kdc[6685](info): commencing operation
krb5kdc: starting...
W20260927 18:37:40.495573 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:40 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:40.604369 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38715
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:38715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:40.707829 6698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:40.708057 6698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:40.708119 6698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:40.711961 6698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:40.712011 6698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:40.712028 6698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:40.712044 6698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:40.712056 6698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:40.717056 6698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:38715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:40.717980 6698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:40.718930 6698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:40.726712 6703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:40.726729 6706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:40.726751 6704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:40.726910 6698 server_base.cc:1034] running on GCE node
I20260927 18:37:40.727542 6698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:40.728579 6698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:40.729732 6698 hybrid_clock.cc:648] HybridClock initialized: now 1790534260729722 us; error 30 us; skew 500 ppm
Sep 27 18:37:40 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:40.734935 6698 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:40.735775 6698 webserver.cc:460] Webserver started at http://127.6.4.62:44109/ using document root <none> and password file <none>
I20260927 18:37:40.736322 6698 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:40.736387 6698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:40.736581 6698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:40.738481 6698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "b744c848ca074cb4a9d3b76dff82ce10"
format_stamp: "Formatted at 2026-09-27 18:37:40 on dist-test-slave-bzzj"
server_key: "08b779fd283c25fd845186c09fc2a35e"
server_key_iv: "c9d69cf96abed6f81aa4d6d0cd42b87f"
server_key_version: "encryptionkey@0"
I20260927 18:37:40.739028 6698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "b744c848ca074cb4a9d3b76dff82ce10"
format_stamp: "Formatted at 2026-09-27 18:37:40 on dist-test-slave-bzzj"
server_key: "08b779fd283c25fd845186c09fc2a35e"
server_key_iv: "c9d69cf96abed6f81aa4d6d0cd42b87f"
server_key_version: "encryptionkey@0"
I20260927 18:37:40.742471 6698 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:40.744721 6713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:40.745867 6698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:37:40.745976 6698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "b744c848ca074cb4a9d3b76dff82ce10"
format_stamp: "Formatted at 2026-09-27 18:37:40 on dist-test-slave-bzzj"
server_key: "08b779fd283c25fd845186c09fc2a35e"
server_key_iv: "c9d69cf96abed6f81aa4d6d0cd42b87f"
server_key_version: "encryptionkey@0"
I20260927 18:37:40.746068 6698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:40.769032 6698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:40.771695 6698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:40.771860 6698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:40.778263 6698 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:38715
I20260927 18:37:40.778285 6765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:38715 every 8 connection(s)
I20260927 18:37:40.779139 6698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:40.779320 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6698
I20260927 18:37:40.779414 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:40.779670 6160 external_mini_cluster.cc:1428] Setting key 229d53d702160fd7ae7baceab5e88974
I20260927 18:37:40.782281 6766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 18:37:40 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:40.788559 6766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10: Bootstrap starting.
I20260927 18:37:40.791127 6766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:40.791967 6766 log.cc:824] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:40.793900 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:40.780996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.6.4.62:38715)
0927 18:37:40.781398 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:40.781415 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:40.781457 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:37:40.782260 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:40.782283 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:40.782690 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:40.783031 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:40.783633 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:40.785135 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:40.785944 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:40.786580 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:40.786738 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:40.789056 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:40.789063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:40.789078 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:40.789107 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:40.790862 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:40.791259 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:40.791265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:40.791270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:40.791411 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:40.791653 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:40.791663 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:40.791664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:40.792192 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:40.792377 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:40.792715 (+ 338us) server_negotiation.cc:299] Negotiation successful
0927 18:37:40.792979 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":108,"threads_started":1}
I20260927 18:37:40.794230 6766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10: No bootstrap required, opened a new log
I20260927 18:37:40.797664 6766 raft_consensus.cc:348] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b744c848ca074cb4a9d3b76dff82ce10" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38715 } }
I20260927 18:37:40.797861 6766 raft_consensus.cc:374] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:40.797899 6766 raft_consensus.cc:729] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b744c848ca074cb4a9d3b76dff82ce10, State: Initialized, Role: FOLLOWER
I20260927 18:37:40.798269 6766 consensus_queue.cc:260] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b744c848ca074cb4a9d3b76dff82ce10" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38715 } }
I20260927 18:37:40.798384 6766 raft_consensus.cc:388] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:40.798429 6766 raft_consensus.cc:482] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:40.798496 6766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:40.799244 6766 raft_consensus.cc:504] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b744c848ca074cb4a9d3b76dff82ce10" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38715 } }
I20260927 18:37:40.799520 6766 leader_election.cc:302] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b744c848ca074cb4a9d3b76dff82ce10; no voters:
I20260927 18:37:40.799753 6766 leader_election.cc:288] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:40.799855 6771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:40.800047 6771 raft_consensus.cc:686] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [term 1 LEADER]: Becoming Leader. State: Replica: b744c848ca074cb4a9d3b76dff82ce10, State: Running, Role: LEADER
I20260927 18:37:40.800417 6771 consensus_queue.cc:237] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b744c848ca074cb4a9d3b76dff82ce10" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38715 } }
I20260927 18:37:40.800910 6766 sys_catalog.cc:564] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:40.801947 6772 sys_catalog.cc:455] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b744c848ca074cb4a9d3b76dff82ce10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b744c848ca074cb4a9d3b76dff82ce10" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38715 } } }
I20260927 18:37:40.802049 6772 sys_catalog.cc:458] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:40.802410 6780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:40.802397 6773 sys_catalog.cc:455] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b744c848ca074cb4a9d3b76dff82ce10. Latest consensus state: current_term: 1 leader_uuid: "b744c848ca074cb4a9d3b76dff82ce10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b744c848ca074cb4a9d3b76dff82ce10" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38715 } } }
I20260927 18:37:40.802589 6773 sys_catalog.cc:458] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:40.805166 6780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:40.810318 6780 catalog_manager.cc:1347] Generated new cluster ID: b6f29a8e508b4477956854d826764d2f
I20260927 18:37:40.810375 6780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:40.839397 6780 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:40.840461 6780 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:40.848636 6780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b744c848ca074cb4a9d3b76dff82ce10: Generated new TSK 0
I20260927 18:37:40.849318 6780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:40.880493 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38715
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:40.986037 6792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:40.986265 6792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:40.986305 6792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:40.990193 6792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:40.990247 6792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:40.990301 6792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:40.995340 6792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38715
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:40.996274 6792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:40.997182 6792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:41.006521 6798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:41.006531 6800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:41.006582 6792 server_base.cc:1034] running on GCE node
W20260927 18:37:41.006528 6797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:41.007061 6792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:41.007634 6792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:41.008817 6792 hybrid_clock.cc:648] HybridClock initialized: now 1790534261008790 us; error 40 us; skew 500 ppm
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:41.011452 6792 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:41.012290 6792 webserver.cc:460] Webserver started at http://127.6.4.1:34469/ using document root <none> and password file <none>
I20260927 18:37:41.012787 6792 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:41.012840 6792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:41.012992 6792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:41.014927 6792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "3c87eaae896d4c8da87051a8de40dcaf"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "e0a32a9da46de36953e85f797cef3ef0"
server_key_iv: "d408b876e028c96965f9a3a8a855355e"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.015445 6792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "3c87eaae896d4c8da87051a8de40dcaf"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "e0a32a9da46de36953e85f797cef3ef0"
server_key_iv: "d408b876e028c96965f9a3a8a855355e"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.018918 6792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:37:41.021175 6807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.022284 6792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:41.022384 6792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "3c87eaae896d4c8da87051a8de40dcaf"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "e0a32a9da46de36953e85f797cef3ef0"
server_key_iv: "d408b876e028c96965f9a3a8a855355e"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.022475 6792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:41.038426 6792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:41.040933 6792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:41.041083 6792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:41.041587 6792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:41.042321 6792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:41.042366 6792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.042403 6792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:41.042433 6792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.050489 6792 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:46485
I20260927 18:37:41.050513 6920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:46485 every 8 connection(s)
I20260927 18:37:41.051262 6792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:37:41.055636 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6792
I20260927 18:37:41.055717 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:41.055954 6160 external_mini_cluster.cc:1428] Setting key ca8900b78e47c94379c2755356c514da
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:41.063755 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.052836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:34383 (local address 127.6.4.62:38715)
0927 18:37:41.052946 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.052949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.053678 (+ 729us) server_negotiation.cc:407] Connection header received
0927 18:37:41.054912 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.054916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.054974 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.055043 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.056181 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.056700 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.057763 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.057981 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.058064 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.060417 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.060421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.060423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.060448 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.061903 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.062348 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.062351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.062354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.062441 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.062754 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.062757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.062759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.062965 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.063066 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.063484 (+ 418us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.063598 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:37:41.064476 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.053140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:38715 (local address 127.6.4.1:34383)
0927 18:37:41.053547 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.053585 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.054719 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.055211 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.055217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.055523 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:41.056013 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.056031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.056811 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.056815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.057644 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.057652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.058063 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.058109 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.058262 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.058785 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:41.058804 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:41.060269 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:41.062011 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:41.062016 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:41.062026 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:41.062255 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:41.062551 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:41.062553 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:41.062554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:41.062670 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:41.063054 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:41.063058 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:41.063259 (+ 201us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.063407 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.063656 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:37:41.065505 6921 heartbeater.cc:347] Connected to a master server at 127.6.4.62:38715
I20260927 18:37:41.065692 6921 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:41.066094 6921 heartbeater.cc:511] Master 127.6.4.62:38715 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:37:41.067456 6730 ts_manager.cc:194] Registered new tserver with Master: 3c87eaae896d4c8da87051a8de40dcaf (127.6.4.1:46485)
I20260927 18:37:41.068562 6730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:34383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:41.081063 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38715
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:41.184726 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:41.184947 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:41.184988 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:41.188867 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:41.188912 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:41.188978 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:41.193938 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38715
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:41.194825 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:41.195757 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:41.204538 6934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:41.204535 6931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:41.204607 6932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:41.204677 6926 server_base.cc:1034] running on GCE node
I20260927 18:37:41.205145 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:41.205706 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:41.206856 6926 hybrid_clock.cc:648] HybridClock initialized: now 1790534261206840 us; error 29 us; skew 500 ppm
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:41.209376 6926 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:41.210125 6926 webserver.cc:460] Webserver started at http://127.6.4.2:35389/ using document root <none> and password file <none>
I20260927 18:37:41.210619 6926 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:41.210680 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:41.210851 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:41.212759 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "9a678d7876194b00a8c9d640da954736"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "b33e2b3e35ab24e5f270106fa94e9a34"
server_key_iv: "6843a87f1111115f8a057567ba2bbdee"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.213267 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "9a678d7876194b00a8c9d640da954736"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "b33e2b3e35ab24e5f270106fa94e9a34"
server_key_iv: "6843a87f1111115f8a057567ba2bbdee"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.216589 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:37:41.218742 6941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.219839 6926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:41.219945 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "9a678d7876194b00a8c9d640da954736"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "b33e2b3e35ab24e5f270106fa94e9a34"
server_key_iv: "6843a87f1111115f8a057567ba2bbdee"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.220028 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:41.241003 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:41.243880 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:41.244037 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:41.244714 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:41.245455 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:41.245498 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.245549 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:41.245577 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.253711 6926 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:33977
I20260927 18:37:41.253750 7054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:33977 every 8 connection(s)
I20260927 18:37:41.254516 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:41.256222 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 6926
I20260927 18:37:41.256330 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:41.256572 6160 external_mini_cluster.cc:1428] Setting key 991401141f810ecfd85a3a458364b01e
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:37:41.267908 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.256315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:41501 (local address 127.6.4.62:38715)
0927 18:37:41.256952 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.256955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.257403 (+ 448us) server_negotiation.cc:407] Connection header received
0927 18:37:41.258920 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.258922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.258964 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.259042 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.260329 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.260840 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.261967 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.262181 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.262264 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.264528 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.264531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.264533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.264554 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.265847 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.266305 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.266308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.266309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.266387 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.266807 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.266810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.266811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.266989 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.267066 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.267632 (+ 566us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.267755 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554}
I20260927 18:37:41.268615 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.256769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:38715 (local address 127.6.4.2:41501)
0927 18:37:41.257263 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.257312 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.258703 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.259236 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.259245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.259680 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:41.260188 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.260201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.260950 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.260953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.261859 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.261867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.262264 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.262313 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.262464 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.262952 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:41.262967 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:41.264380 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:41.265955 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:41.265959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:41.265970 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:41.266204 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:41.266496 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:41.266499 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:41.266500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:41.266715 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:41.267096 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:41.267102 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:41.267377 (+ 275us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.267562 (+ 185us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.267817 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":147,"threads_started":1}
I20260927 18:37:41.269630 7055 heartbeater.cc:347] Connected to a master server at 127.6.4.62:38715
I20260927 18:37:41.269834 7055 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:41.270260 7055 heartbeater.cc:511] Master 127.6.4.62:38715 requested a full tablet report, sending...
I20260927 18:37:41.271206 6730 ts_manager.cc:194] Registered new tserver with Master: 9a678d7876194b00a8c9d640da954736 (127.6.4.2:33977)
I20260927 18:37:41.271956 6730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:41501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:41.282814 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38715
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:41.386349 7060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:41.386566 7060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:41.386605 7060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:41.390463 7060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:41.390511 7060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:41.390569 7060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:41.395421 7060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38715
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:41.396292 7060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:41.397185 7060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:37:41.406134 7060 server_base.cc:1034] running on GCE node
W20260927 18:37:41.406090 7068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:41.406068 7066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:41.406100 7065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:41.406694 7060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:41.407238 7060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:41.408386 7060 hybrid_clock.cc:648] HybridClock initialized: now 1790534261408371 us; error 27 us; skew 500 ppm
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:41.410904 7060 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:41.411880 7060 webserver.cc:460] Webserver started at http://127.6.4.3:45811/ using document root <none> and password file <none>
I20260927 18:37:41.412418 7060 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:41.412475 7060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:41.412631 7060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:41.414508 7060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "0188aab880db4b43b4f28cef9fdb220b"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "bcde103efa950cff81c22e767b94942f"
server_key_iv: "68852bb3f18995c475e5e114d6c13fd1"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.414999 7060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "0188aab880db4b43b4f28cef9fdb220b"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "bcde103efa950cff81c22e767b94942f"
server_key_iv: "68852bb3f18995c475e5e114d6c13fd1"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.418286 7060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:41.420513 7075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.421648 7060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:37:41.421753 7060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "0188aab880db4b43b4f28cef9fdb220b"
format_stamp: "Formatted at 2026-09-27 18:37:41 on dist-test-slave-bzzj"
server_key: "bcde103efa950cff81c22e767b94942f"
server_key_iv: "68852bb3f18995c475e5e114d6c13fd1"
server_key_version: "encryptionkey@0"
I20260927 18:37:41.421839 7060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:41.444917 7060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:41.447954 7060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:41.448146 7060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:41.448714 7060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:41.449525 7060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:41.449569 7060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.449620 7060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:41.449647 7060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:41.457347 7060 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37089
I20260927 18:37:41.457397 7188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37089 every 8 connection(s)
I20260927 18:37:41.458194 7060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:41.467929 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7060
I20260927 18:37:41.468041 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:41.468338 6160 external_mini_cluster.cc:1428] Setting key 96f43a14d0bf26d5abe8045c51bebe05
I20260927 18:37:41.470783 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.459829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:37067 (local address 127.6.4.62:38715)
0927 18:37:41.459958 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.459962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.460760 (+ 798us) server_negotiation.cc:407] Connection header received
0927 18:37:41.462047 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.462050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.462106 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.462177 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.463309 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.463829 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.464958 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.465175 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.465261 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.467447 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.467450 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.467452 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.467475 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.468843 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.469335 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.469338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.469340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.469423 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.469764 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.469768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.469770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.469971 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.470054 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.470517 (+ 463us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.470632 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 18:37:41.471542 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.460206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:38715 (local address 127.6.4.3:37067)
0927 18:37:41.460618 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.460657 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.461847 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.462375 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.462382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.462717 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:41.463171 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.463184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.463943 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.463947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.464843 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.464852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.465260 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.465310 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.465466 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.465976 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:41.465991 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:41.467295 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:41.468938 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:41.468943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:41.468959 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:41.469252 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:41.469501 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:41.469504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:41.469506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:41.469682 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:41.470062 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:41.470068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:41.470299 (+ 231us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.470458 (+ 159us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.470776 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":109,"threads_started":1}
I20260927 18:37:41.472585 7189 heartbeater.cc:347] Connected to a master server at 127.6.4.62:38715
I20260927 18:37:41.472800 7189 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:41.473243 7189 heartbeater.cc:511] Master 127.6.4.62:38715 requested a full tablet report, sending...
I20260927 18:37:41.474211 6730 ts_manager.cc:194] Registered new tserver with Master: 0188aab880db4b43b4f28cef9fdb220b (127.6.4.3:37089)
I20260927 18:37:41.474745 6730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:37067
I20260927 18:37:41.481770 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:37:41.491204 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.484947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.6.4.62:38715)
0927 18:37:41.485040 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.485044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.485201 (+ 157us) server_negotiation.cc:407] Connection header received
0927 18:37:41.485298 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.485300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.485333 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.485392 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.485998 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.486486 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.487287 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.487516 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.487624 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.488809 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.488812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.488814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.488836 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.489944 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.490281 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.490283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.490284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.490353 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.490564 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.490566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.490567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.490753 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.490844 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.491011 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.491097 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:37:41.495388 6730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:37:41.498283 6730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:41.511658 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.506705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.6.4.1:46485)
0927 18:37:41.507331 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.507345 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.507367 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:37:41.507438 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.507443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.507597 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.507863 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:41.508028 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.508827 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.510208 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.510991 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.511087 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.511232 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.511413 (+ 181us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.511507 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":145,"threads_started":1}
I20260927 18:37:41.511653 7200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.506055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:46485 (local address 127.0.0.1:43096)
0927 18:37:41.506500 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.506545 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.506665 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.507748 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.507751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.507778 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:41.507938 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.507945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.508941 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.508945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.510068 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.510078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.511107 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.511111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.511221 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.511261 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.511371 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.511491 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":17,"spinlock_wait_cycles":5632,"thread_start_us":68,"threads_started":1}
I20260927 18:37:41.512159 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.506187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33977 (local address 127.0.0.1:41304)
0927 18:37:41.507114 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.507127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.507209 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.507736 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.507742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.507776 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:41.508152 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.508166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.509515 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.509518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.510625 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.510634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.511757 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.511761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.511852 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.511874 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.511935 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.511994 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":48,"spinlock_wait_cycles":5760,"thread_start_us":102,"threads_started":1}
I20260927 18:37:41.512174 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.506865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.6.4.3:37089)
0927 18:37:41.507598 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.507608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.507629 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:37:41.507696 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.507701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.507853 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.507972 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:41.508220 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.509391 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.510716 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.511328 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.511718 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.511862 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.511911 (+ 49us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.511999 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":127,"threads_started":1}
I20260927 18:37:41.512205 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.506706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37089 (local address 127.0.0.1:60764)
0927 18:37:41.507306 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.507317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.507383 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.507973 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.507976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.507992 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:41.508089 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.508116 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.509524 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.509528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.510604 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.510613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.511442 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.511445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.511521 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.511543 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.511587 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.511641 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":128,"thread_start_us":80,"threads_started":1}
I20260927 18:37:41.512529 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.506331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.6.4.2:33977)
0927 18:37:41.507156 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.507170 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.507193 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:37:41.507349 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.507355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.507561 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.507722 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:41.508249 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.509372 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.510761 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.511471 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.511561 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.511696 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.512306 (+ 610us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.512397 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":110,"threads_started":1}
I20260927 18:37:41.514789 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 5a462ce0ff20480d940903e985889a0f (DEFAULT_TABLE table=test-table [id=d6ab25e09ad947e3a38741a68a63413a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:41.514896 6855 tablet_service.cc:1467] Processing CreateTablet for tablet 5a462ce0ff20480d940903e985889a0f (DEFAULT_TABLE table=test-table [id=d6ab25e09ad947e3a38741a68a63413a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:41.514894 6989 tablet_service.cc:1467] Processing CreateTablet for tablet 5a462ce0ff20480d940903e985889a0f (DEFAULT_TABLE table=test-table [id=d6ab25e09ad947e3a38741a68a63413a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:41.515698 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a462ce0ff20480d940903e985889a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:41.515848 6855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a462ce0ff20480d940903e985889a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:41.515848 6989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a462ce0ff20480d940903e985889a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:41.522061 7206 tablet_bootstrap.cc:492] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b: Bootstrap starting.
I20260927 18:37:41.522846 7208 tablet_bootstrap.cc:492] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736: Bootstrap starting.
I20260927 18:37:41.524039 7206 tablet_bootstrap.cc:654] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:41.524653 7208 tablet_bootstrap.cc:654] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:41.525127 7206 log.cc:824] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:41.525719 7208 log.cc:824] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:41.526006 7207 tablet_bootstrap.cc:492] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: Bootstrap starting.
I20260927 18:37:41.527007 7206 tablet_bootstrap.cc:492] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b: No bootstrap required, opened a new log
I20260927 18:37:41.527189 7206 ts_tablet_manager.cc:1397] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260927 18:37:41.527799 7207 tablet_bootstrap.cc:654] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:41.527963 7208 tablet_bootstrap.cc:492] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736: No bootstrap required, opened a new log
I20260927 18:37:41.528174 7208 ts_tablet_manager.cc:1397] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:37:41.528786 7207 log.cc:824] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:41.530189 7207 tablet_bootstrap.cc:492] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: No bootstrap required, opened a new log
I20260927 18:37:41.530346 7207 ts_tablet_manager.cc:1397] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260927 18:37:41.530970 7206 raft_consensus.cc:348] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.531226 7206 raft_consensus.cc:374] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:41.531280 7206 raft_consensus.cc:729] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0188aab880db4b43b4f28cef9fdb220b, State: Initialized, Role: FOLLOWER
I20260927 18:37:41.531705 7206 consensus_queue.cc:260] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.532084 7208 raft_consensus.cc:348] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.532327 7189 heartbeater.cc:503] Master 127.6.4.62:38715 was elected leader, sending a full tablet report...
I20260927 18:37:41.532351 7208 raft_consensus.cc:374] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:41.532404 7208 raft_consensus.cc:729] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a678d7876194b00a8c9d640da954736, State: Initialized, Role: FOLLOWER
I20260927 18:37:41.532823 7208 consensus_queue.cc:260] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.532918 7206 ts_tablet_manager.cc:1428] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260927 18:37:41.533317 7207 raft_consensus.cc:348] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.533689 7208 ts_tablet_manager.cc:1428] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260927 18:37:41.533564 7207 raft_consensus.cc:374] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:41.533838 7207 raft_consensus.cc:729] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c87eaae896d4c8da87051a8de40dcaf, State: Initialized, Role: FOLLOWER
I20260927 18:37:41.533962 7055 heartbeater.cc:503] Master 127.6.4.62:38715 was elected leader, sending a full tablet report...
I20260927 18:37:41.534212 7207 consensus_queue.cc:260] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.534811 6921 heartbeater.cc:503] Master 127.6.4.62:38715 was elected leader, sending a full tablet report...
I20260927 18:37:41.535054 7207 ts_tablet_manager.cc:1428] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260927 18:37:41.552291 6922 tablet.cc:2369] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:37:41.597051 7213 raft_consensus.cc:482] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:41.597273 7213 raft_consensus.cc:504] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.598387 7213 leader_election.cc:288] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c87eaae896d4c8da87051a8de40dcaf (127.6.4.1:46485), 0188aab880db4b43b4f28cef9fdb220b (127.6.4.3:37089)
I20260927 18:37:41.602583 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.598684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:34217 (local address 127.6.4.1:46485)
0927 18:37:41.598924 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.598928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.598972 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:37:41.599027 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.599030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.599095 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.599179 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:41.599510 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.600343 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.601622 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.602088 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.602201 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.602267 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.602419 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.602488 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260927 18:37:41.602583 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.598602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:46485 (local address 127.6.4.2:34217)
0927 18:37:41.598802 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.598817 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.598906 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.599194 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.599196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.599211 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:41.599372 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.599379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.600434 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.600438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.601510 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.601519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.602198 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.602201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.602297 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.602318 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.602383 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.602456 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":21}
I20260927 18:37:41.602804 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.598719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:47279 (local address 127.6.4.3:37089)
0927 18:37:41.598841 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.598846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.599008 (+ 162us) server_negotiation.cc:407] Connection header received
0927 18:37:41.599113 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.599116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.599160 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.599227 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:41.599550 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.600326 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.601710 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.602165 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.602265 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.602325 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.602631 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.602698 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:37:41.603154 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.598625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37089 (local address 127.6.4.2:47279)
0927 18:37:41.598915 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:41.598926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:41.599023 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:41.599249 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:41.599251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:41.599262 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:41.599376 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.599381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.600611 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.600613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.601593 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:41.601603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.602380 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.602384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:41.602489 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.602523 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 18:37:41.602933 (+ 410us) client_negotiation.cc:240] Negotiation successful
0927 18:37:41.603010 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":21,"thread_start_us":100,"threads_started":1}
I20260927 18:37:41.603288 6875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a462ce0ff20480d940903e985889a0f" candidate_uuid: "9a678d7876194b00a8c9d640da954736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c87eaae896d4c8da87051a8de40dcaf" is_pre_election: true
I20260927 18:37:41.603650 6875 raft_consensus.cc:2454] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a678d7876194b00a8c9d640da954736 in term 0.
I20260927 18:37:41.604111 6944 leader_election.cc:302] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c87eaae896d4c8da87051a8de40dcaf, 9a678d7876194b00a8c9d640da954736; no voters:
I20260927 18:37:41.604370 7213 raft_consensus.cc:2781] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:41.604393 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a462ce0ff20480d940903e985889a0f" candidate_uuid: "9a678d7876194b00a8c9d640da954736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0188aab880db4b43b4f28cef9fdb220b" is_pre_election: true
I20260927 18:37:41.604467 7213 raft_consensus.cc:482] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:41.604513 7213 raft_consensus.cc:3037] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:41.604735 7143 raft_consensus.cc:2454] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a678d7876194b00a8c9d640da954736 in term 0.
I20260927 18:37:41.605686 7213 raft_consensus.cc:504] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.605959 7213 leader_election.cc:288] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [CANDIDATE]: Term 1 election: Requested vote from peers 3c87eaae896d4c8da87051a8de40dcaf (127.6.4.1:46485), 0188aab880db4b43b4f28cef9fdb220b (127.6.4.3:37089)
I20260927 18:37:41.606263 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a462ce0ff20480d940903e985889a0f" candidate_uuid: "9a678d7876194b00a8c9d640da954736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0188aab880db4b43b4f28cef9fdb220b"
I20260927 18:37:41.606253 6875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a462ce0ff20480d940903e985889a0f" candidate_uuid: "9a678d7876194b00a8c9d640da954736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c87eaae896d4c8da87051a8de40dcaf"
I20260927 18:37:41.606367 7143 raft_consensus.cc:3037] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:41.606367 6875 raft_consensus.cc:3037] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:41.607535 6875 raft_consensus.cc:2454] T 5a462ce0ff20480d940903e985889a0f P 3c87eaae896d4c8da87051a8de40dcaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a678d7876194b00a8c9d640da954736 in term 1.
I20260927 18:37:41.607575 7143 raft_consensus.cc:2454] T 5a462ce0ff20480d940903e985889a0f P 0188aab880db4b43b4f28cef9fdb220b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a678d7876194b00a8c9d640da954736 in term 1.
I20260927 18:37:41.607832 6944 leader_election.cc:302] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c87eaae896d4c8da87051a8de40dcaf, 9a678d7876194b00a8c9d640da954736; no voters:
I20260927 18:37:41.608014 7213 raft_consensus.cc:2781] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:41.608256 7213 raft_consensus.cc:686] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [term 1 LEADER]: Becoming Leader. State: Replica: 9a678d7876194b00a8c9d640da954736, State: Running, Role: LEADER
I20260927 18:37:41.608549 7213 consensus_queue.cc:237] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } }
I20260927 18:37:41.611541 6728 catalog_manager.cc:5495] T 5a462ce0ff20480d940903e985889a0f P 9a678d7876194b00a8c9d640da954736 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a678d7876194b00a8c9d640da954736 (127.6.4.2). New cstate: current_term: 1 leader_uuid: "9a678d7876194b00a8c9d640da954736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c87eaae896d4c8da87051a8de40dcaf" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a678d7876194b00a8c9d640da954736" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0188aab880db4b43b4f28cef9fdb220b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37089 } health_report { overall_health: UNKNOWN } } }
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:41.634091 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.626158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.6.4.1:46485)
0927 18:37:41.626262 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.626267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.626504 (+ 237us) server_negotiation.cc:407] Connection header received
0927 18:37:41.626611 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.626615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.626662 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.626729 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.627416 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.627911 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.628768 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.628983 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.629068 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.631336 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.631347 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.631361 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.631383 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.632732 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.633087 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.633093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.633094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.633179 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.633364 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.633365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.633367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.633599 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.633749 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.633889 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.633976 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:41.658489 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.651067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.6.4.1:46485)
0927 18:37:41.651199 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.651204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.651326 (+ 122us) server_negotiation.cc:407] Connection header received
0927 18:37:41.651456 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.651460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.651499 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.651562 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.652229 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.652748 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.653550 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.653775 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.653858 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.656001 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.656004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.656006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.656029 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.657181 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.657552 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.657554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.657556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.657623 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.657834 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.657837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.657838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.657988 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.658086 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.658284 (+ 198us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.658375 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[6685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:41.681103 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:41.673861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.6.4.1:46485)
0927 18:37:41.673967 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:41.673970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:41.674117 (+ 147us) server_negotiation.cc:407] Connection header received
0927 18:37:41.674237 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:41.674239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:41.674275 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:41.674338 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:41.674899 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.675386 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.676185 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:41.676380 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:41.676449 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:41.678513 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:41.678517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:41.678520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:41.678544 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:41.679802 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.680177 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.680179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.680180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.680251 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:41.680446 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:41.680448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:41.680449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:41.680585 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:41.680676 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:41.680851 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 18:37:41.680949 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:37:41.682469 6835 tablet_service.cc:2908] Rejecting scan request for tablet 5a462ce0ff20480d940903e985889a0f: Uninitialized: clean time has not yet been initialized
I20260927 18:37:41.684327 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6792
I20260927 18:37:41.689890 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6926
I20260927 18:37:41.694387 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7060
I20260927 18:37:41.698580 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 6698
2026-09-27T18:37:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2149 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[7230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[7230](info): set up 2 sockets
Sep 27 18:37:41 dist-test-slave-bzzj krb5kdc[7230](info): commencing operation
krb5kdc: starting...
W20260927 18:37:42.643627 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:42 dist-test-slave-bzzj krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:42.755198 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33545
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:33545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:42.859606 7243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:42.859853 7243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:42.859905 7243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:42.863749 7243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:42.863797 7243 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:37:42.863824 7243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:42.863847 7243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:42.863869 7243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:42.863896 7243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:42.868882 7243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:33545
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:42.869820 7243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:42.870757 7243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:42.878614 7248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:42.878641 7251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:42.878661 7249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:42.878830 7243 server_base.cc:1034] running on GCE node
I20260927 18:37:42.879462 7243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:42.880553 7243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:42.881721 7243 hybrid_clock.cc:648] HybridClock initialized: now 1790534262881708 us; error 28 us; skew 500 ppm
Sep 27 18:37:42 dist-test-slave-bzzj krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534262, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:42.884591 7243 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:42.885389 7243 webserver.cc:460] Webserver started at http://127.6.4.62:39389/ using document root <none> and password file <none>
I20260927 18:37:42.885885 7243 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:42.885946 7243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:42.886111 7243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:42.888253 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "55a2db856edd49e09711882a061d813f"
format_stamp: "Formatted at 2026-09-27 18:37:42 on dist-test-slave-bzzj"
server_key: "0cd451e44add6e314977806a815c9f32"
server_key_iv: "6ee16d1e7d28fa633ad5520352525640"
server_key_version: "encryptionkey@0"
I20260927 18:37:42.888770 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "55a2db856edd49e09711882a061d813f"
format_stamp: "Formatted at 2026-09-27 18:37:42 on dist-test-slave-bzzj"
server_key: "0cd451e44add6e314977806a815c9f32"
server_key_iv: "6ee16d1e7d28fa633ad5520352525640"
server_key_version: "encryptionkey@0"
I20260927 18:37:42.892318 7243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:42.894527 7258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:42.895699 7243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:42.895792 7243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "55a2db856edd49e09711882a061d813f"
format_stamp: "Formatted at 2026-09-27 18:37:42 on dist-test-slave-bzzj"
server_key: "0cd451e44add6e314977806a815c9f32"
server_key_iv: "6ee16d1e7d28fa633ad5520352525640"
server_key_version: "encryptionkey@0"
I20260927 18:37:42.895875 7243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:42.906443 7243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:42.909384 7243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:42.909524 7243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:42.916519 7243 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:33545
I20260927 18:37:42.916559 7320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:33545 every 8 connection(s)
I20260927 18:37:42.917361 7243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:42.919888 7321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:42.920018 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7243
I20260927 18:37:42.920156 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:42.920398 6160 external_mini_cluster.cc:1428] Setting key 26fe7bce60f7441b635daa40ab76b518
I20260927 18:37:42.925666 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: Bootstrap starting.
I20260927 18:37:42.927695 7321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: Neither blocks nor log segments found. Creating new log.
Sep 27 18:37:42 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:42.928412 7321 log.cc:824] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:42.930066 7321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: No bootstrap required, opened a new log
I20260927 18:37:42.932837 7321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2db856edd49e09711882a061d813f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33545 } }
I20260927 18:37:42.933045 7321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:42.933100 7321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a2db856edd49e09711882a061d813f, State: Initialized, Role: FOLLOWER
I20260927 18:37:42.933271 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:42.921641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.6.4.62:33545)
0927 18:37:42.922083 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:42.922097 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:42.922130 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:37:42.922821 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:42.922838 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:42.923150 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:42.923450 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:42.924199 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:42.925147 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:42.925964 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:42.926418 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:42.926579 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:42.928768 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:42.928772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:42.928784 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:42.928810 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:42.930696 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:42.931084 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:42.931089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:42.931093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:42.931193 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:42.931443 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:42.931446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:42.931448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:42.931806 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:42.931982 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:42.932304 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 18:37:42.932559 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":135,"threads_started":1}
I20260927 18:37:42.933547 7321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2db856edd49e09711882a061d813f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33545 } }
I20260927 18:37:42.933663 7321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:42.933728 7321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:42.933822 7321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:42.934727 7321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2db856edd49e09711882a061d813f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33545 } }
I20260927 18:37:42.935050 7321 leader_election.cc:302] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55a2db856edd49e09711882a061d813f; no voters:
I20260927 18:37:42.935335 7321 leader_election.cc:288] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:42.935496 7326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:42.935686 7326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [term 1 LEADER]: Becoming Leader. State: Replica: 55a2db856edd49e09711882a061d813f, State: Running, Role: LEADER
I20260927 18:37:42.935954 7326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2db856edd49e09711882a061d813f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33545 } }
I20260927 18:37:42.936447 7321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:42.937487 7328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a2db856edd49e09711882a061d813f. Latest consensus state: current_term: 1 leader_uuid: "55a2db856edd49e09711882a061d813f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2db856edd49e09711882a061d813f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33545 } } }
I20260927 18:37:42.937608 7328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:42.937841 7327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55a2db856edd49e09711882a061d813f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2db856edd49e09711882a061d813f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33545 } } }
I20260927 18:37:42.937929 7327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:42.937929 7335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:42.940840 7335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:42.945652 7335 catalog_manager.cc:1347] Generated new cluster ID: 9e5a4c32c27c4476838b13e0064f366c
I20260927 18:37:42.945719 7335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:42.982245 7335 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:42.983487 7335 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:42.991187 7335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: Generated new TSK 0
I20260927 18:37:42.991982 7335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:43.027307 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33545
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:43.131111 7347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:43.131332 7347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:43.131371 7347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:43.135279 7347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:43.135330 7347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:43.135370 7347 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:37:43.135419 7347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:43.140555 7347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:33545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:43.141495 7347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:43.142417 7347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:43.151235 7353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:43.151245 7352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:43.151333 7355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:43.151451 7347 server_base.cc:1034] running on GCE node
I20260927 18:37:43.152015 7347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:43.152616 7347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:43.153782 7347 hybrid_clock.cc:648] HybridClock initialized: now 1790534263153763 us; error 27 us; skew 500 ppm
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:43.156407 7347 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:43.157218 7347 webserver.cc:460] Webserver started at http://127.6.4.1:43637/ using document root <none> and password file <none>
I20260927 18:37:43.157735 7347 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:43.157799 7347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:43.157979 7347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:43.159900 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "3397eb3abd4c4655a47fb0ec9d3f8571"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "9e5e69394000f2414f7b185bb1de9d3f"
server_key_iv: "9da47556a697ca4ad0446df568a6d01f"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.160519 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "3397eb3abd4c4655a47fb0ec9d3f8571"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "9e5e69394000f2414f7b185bb1de9d3f"
server_key_iv: "9da47556a697ca4ad0446df568a6d01f"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.164794 7347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 18:37:43.166992 7362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:43.168231 7347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:37:43.168326 7347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "3397eb3abd4c4655a47fb0ec9d3f8571"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "9e5e69394000f2414f7b185bb1de9d3f"
server_key_iv: "9da47556a697ca4ad0446df568a6d01f"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.168406 7347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:43.185814 7347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:43.188565 7347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:43.188704 7347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:43.189975 7347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:43.190021 7347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:43.190058 7347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:43.190073 7347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:43.199427 7347 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:39537
I20260927 18:37:43.199551 7477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:39537 every 8 connection(s)
I20260927 18:37:43.200368 7347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:37:43.202541 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.190701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:46203 (local address 127.6.4.62:33545)
0927 18:37:43.190850 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.190854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.191646 (+ 792us) server_negotiation.cc:407] Connection header received
0927 18:37:43.192752 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.192755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.192812 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.192902 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.194172 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.194968 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.196293 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.196644 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.196760 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.199215 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.199219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.199222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.199250 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.200738 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.201276 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.201279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.201280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.201356 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.201690 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.201692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.201693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.201849 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.201917 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.202303 (+ 386us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.202393 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 18:37:43.202672 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7347
I20260927 18:37:43.202773 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:43.203012 6160 external_mini_cluster.cc:1428] Setting key b47443136a2ad86b655132719bf4b715
I20260927 18:37:43.203320 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.191052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.6.4.1:46203)
0927 18:37:43.191483 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.191540 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.192546 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.193110 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.193119 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.193464 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:43.194035 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.194050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.195109 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.195113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.196192 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.196202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.196745 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.196799 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.196961 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.197464 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:43.197480 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:43.199056 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:43.200863 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.200869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.200881 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.201180 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.201433 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.201435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.201437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.201594 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.201922 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:43.201926 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:43.202128 (+ 202us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.202267 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.202479 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
I20260927 18:37:43.205765 7478 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33545
I20260927 18:37:43.205909 7478 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:43.206349 7478 heartbeater.cc:511] Master 127.6.4.62:33545 requested a full tablet report, sending...
I20260927 18:37:43.207949 7275 ts_manager.cc:194] Registered new tserver with Master: 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1:39537)
I20260927 18:37:43.209383 7275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:46203
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:37:43.213629 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.206480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:55139 (local address 127.6.4.62:33545)
0927 18:37:43.206608 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.206611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.206892 (+ 281us) server_negotiation.cc:407] Connection header received
0927 18:37:43.206989 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.206992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.207036 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.207105 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.207611 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.208385 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.209516 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.209840 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.209952 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.210930 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.210934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.210936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.210960 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.212177 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.212571 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.212574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.212576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.212647 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.212887 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.212890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.212892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.213044 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.213124 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.213371 (+ 247us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.213469 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:37:43.213637 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.206480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.6.4.1:55139)
0927 18:37:43.206803 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.206813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.206908 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.207201 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.207203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.207390 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:43.207500 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.207507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.208516 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.208519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.209377 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.209387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.209938 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.209954 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.210043 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.210497 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:43.210509 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:43.210803 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:43.212269 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.212272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.212274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.212487 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.212732 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.212735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.212737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.212812 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.213129 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:43.213132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:43.213223 (+ 91us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.213313 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.213434 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:43.228762 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33545
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:43.332196 7487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:43.332423 7487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:43.332479 7487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:43.336371 7487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:43.336423 7487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:43.336474 7487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:37:43.336520 7487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:43.341536 7487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:33545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:43.342425 7487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:43.343338 7487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:43.351966 7495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:43.352044 7492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:43.351962 7493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:43.352208 7487 server_base.cc:1034] running on GCE node
I20260927 18:37:43.352650 7487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:43.353216 7487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:43.354360 7487 hybrid_clock.cc:648] HybridClock initialized: now 1790534263354335 us; error 38 us; skew 500 ppm
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:43.356809 7487 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:43.357658 7487 webserver.cc:460] Webserver started at http://127.6.4.2:40443/ using document root <none> and password file <none>
I20260927 18:37:43.358155 7487 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:43.358217 7487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:43.358397 7487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:43.360404 7487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "af2963a8c9e54b37a1246a3c492d837d"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "8ce6829a2b47ecb836a83d71e859b3db"
server_key_iv: "cd7a683f3b06dbb23f3f1d143334873d"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.360922 7487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "af2963a8c9e54b37a1246a3c492d837d"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "8ce6829a2b47ecb836a83d71e859b3db"
server_key_iv: "cd7a683f3b06dbb23f3f1d143334873d"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.364400 7487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:43.366618 7502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:43.367738 7487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:43.367836 7487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "af2963a8c9e54b37a1246a3c492d837d"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "8ce6829a2b47ecb836a83d71e859b3db"
server_key_iv: "cd7a683f3b06dbb23f3f1d143334873d"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.367919 7487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:43.388379 7487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:43.390899 7487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:43.391057 7487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:43.392402 7487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:43.392450 7487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:43.392508 7487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:43.392534 7487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:43.402467 7487 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:44655
I20260927 18:37:43.402539 7617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:44655 every 8 connection(s)
I20260927 18:37:43.403281 7487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:43.404222 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7487
I20260927 18:37:43.404316 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:43.404579 6160 external_mini_cluster.cc:1428] Setting key a6cca8b0016dc6921c82175bc27399f1
I20260927 18:37:43.405210 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.393103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51317 (local address 127.6.4.62:33545)
0927 18:37:43.393225 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.393228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.394215 (+ 987us) server_negotiation.cc:407] Connection header received
0927 18:37:43.395149 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.395153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.395207 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.395261 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.396656 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.397326 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.398658 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.398978 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.399078 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.401690 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.401693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.401696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.401721 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.403012 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.403547 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.403551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.403553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.403626 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.404007 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.404010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.404012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.404234 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.404329 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.404921 (+ 592us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.405040 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:37:43.406024 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.393627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.6.4.2:51317)
0927 18:37:43.394070 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.394121 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.394921 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.395637 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.395646 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.395993 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:43.396512 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.396531 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.397439 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.397443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.398553 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.398562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.399116 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.399167 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.399323 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.399886 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:43.399905 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:43.401527 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:43.403118 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.403123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.403136 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.403460 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.403736 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.403739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.403740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.403916 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.404382 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:43.404388 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:43.404696 (+ 308us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.404860 (+ 164us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.405160 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":129,"threads_started":1}
I20260927 18:37:43.407155 7618 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33545
I20260927 18:37:43.407281 7618 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:43.407819 7618 heartbeater.cc:511] Master 127.6.4.62:33545 requested a full tablet report, sending...
I20260927 18:37:43.408883 7273 ts_manager.cc:194] Registered new tserver with Master: af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655)
I20260927 18:37:43.409531 7273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:51317
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
I20260927 18:37:43.415907 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.409261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51401 (local address 127.6.4.62:33545)
0927 18:37:43.409379 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.409383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.409701 (+ 318us) server_negotiation.cc:407] Connection header received
0927 18:37:43.409812 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.409815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.409860 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.409937 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.410411 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.411161 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.411957 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.412295 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.412401 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.413316 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.413320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.413322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.413348 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.414638 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.414983 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.414985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.414986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.415051 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.415261 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.415263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.415264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.415469 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.415544 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.415729 (+ 185us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.415808 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:37:43.415951 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.409375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.6.4.2:51401)
0927 18:37:43.409626 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.409639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.409729 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.409966 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.409969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.410162 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:43.410292 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.410299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.411294 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.411298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.411850 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.411859 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.412383 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.412396 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.412461 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.412891 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:43.412901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:43.413183 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:43.414739 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.414742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.414743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.414910 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.415126 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.415128 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.415129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.415201 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.415545 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:43.415547 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:43.415614 (+ 67us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.415696 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.415787 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:43.430459 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33545
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:43.535844 7627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:43.536070 7627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:43.536137 7627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:43.540117 7627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:43.540174 7627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:43.540211 7627 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:37:43.540256 7627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:43.545182 7627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:33545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:43.546070 7627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:43.546976 7627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:43.555940 7635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:43.555946 7632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:43.556084 7627 server_base.cc:1034] running on GCE node
W20260927 18:37:43.556005 7633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:43.556533 7627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:43.557094 7627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:43.558236 7627 hybrid_clock.cc:648] HybridClock initialized: now 1790534263558222 us; error 34 us; skew 500 ppm
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:43.560734 7627 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:43.561517 7627 webserver.cc:460] Webserver started at http://127.6.4.3:33377/ using document root <none> and password file <none>
I20260927 18:37:43.562004 7627 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:43.562065 7627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:43.562244 7627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:43.564086 7627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "f690855d58814e41b5bf0e2478e2baf7"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "b5e01aecde982d12f04786809a8711b1"
server_key_iv: "6b3c4f1b0be6288f6ded983b63661d86"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.564643 7627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "f690855d58814e41b5bf0e2478e2baf7"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "b5e01aecde982d12f04786809a8711b1"
server_key_iv: "6b3c4f1b0be6288f6ded983b63661d86"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.567910 7627 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:37:43.570185 7642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:43.571398 7627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:43.571498 7627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "f690855d58814e41b5bf0e2478e2baf7"
format_stamp: "Formatted at 2026-09-27 18:37:43 on dist-test-slave-bzzj"
server_key: "b5e01aecde982d12f04786809a8711b1"
server_key_iv: "6b3c4f1b0be6288f6ded983b63661d86"
server_key_version: "encryptionkey@0"
I20260927 18:37:43.571599 7627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:43.581599 7627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:43.584028 7627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:43.584203 7627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:43.585520 7627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:43.585565 7627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:43.585613 7627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:43.585641 7627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:43.595752 7627 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:42599
I20260927 18:37:43.595783 7757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:42599 every 8 connection(s)
I20260927 18:37:43.596745 7627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:37:43.598006 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.586191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:34271 (local address 127.6.4.62:33545)
0927 18:37:43.586325 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.586330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.587224 (+ 894us) server_negotiation.cc:407] Connection header received
0927 18:37:43.588401 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.588405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.588466 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.588525 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.589776 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.590406 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.591576 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.591799 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.591886 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.594454 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.594456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.594458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.594482 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.595923 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.596407 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.596411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.596412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.596502 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.596824 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.596828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.596829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.597042 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.597147 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.597725 (+ 578us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.597845 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:37:43.598698 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.586643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.6.4.3:34271)
0927 18:37:43.587084 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.587121 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.588154 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.588870 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.588876 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.589173 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:43.589625 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.589644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.590508 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.590512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.591453 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.591463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.591899 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.591950 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.592153 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.592683 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:43.592703 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:43.594294 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:43.596019 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.596023 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.596034 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.596311 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.596583 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.596585 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.596586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.596717 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.597178 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:43.597184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:43.597446 (+ 262us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.597636 (+ 190us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.597926 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":118,"threads_started":1}
I20260927 18:37:43.599828 7758 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33545
I20260927 18:37:43.599957 7758 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:43.600565 7758 heartbeater.cc:511] Master 127.6.4.62:33545 requested a full tablet report, sending...
I20260927 18:37:43.601545 7275 ts_manager.cc:194] Registered new tserver with Master: f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599)
I20260927 18:37:43.602149 7275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:34271
I20260927 18:37:43.605324 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7627
I20260927 18:37:43.605404 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:43.605602 6160 external_mini_cluster.cc:1428] Setting key 9fca30c6f4b20738da6dacaab0ad3b9b
I20260927 18:37:43.607470 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:37:43.609014 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.601953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:60763 (local address 127.6.4.62:33545)
0927 18:37:43.602062 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.602066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.602259 (+ 193us) server_negotiation.cc:407] Connection header received
0927 18:37:43.602355 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.602358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.602398 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.602458 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.602916 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.603387 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.604488 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.604691 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.604770 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.606355 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.606359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.606362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.606386 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.607639 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.608040 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.608042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.608043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.608137 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.608389 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.608392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.608394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.608544 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.608612 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.608835 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.608914 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:37:43.609076 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.601864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.6.4.3:60763)
0927 18:37:43.602177 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.602188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.602272 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.602486 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.602489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.602699 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:43.602804 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.602811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.603560 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.603563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.604397 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.604406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.604771 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.604784 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.604847 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.605888 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:43.605899 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:43.606235 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:43.607736 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.607739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.607741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.607961 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.608223 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:43.608226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:43.608228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:43.608308 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:43.608624 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:43.608626 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:43.608706 (+ 80us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.608804 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.608918 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":86,"threads_started":1}
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:37:43 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:43.631234 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.624153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.6.4.62:33545)
0927 18:37:43.624273 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.624277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.624290 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:37:43.624331 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.624333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.624372 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.624440 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:43.625046 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.625550 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.626311 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.626513 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.626580 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.628716 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:43.628720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:43.628722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:43.628745 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:43.629979 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.630314 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.630316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.630318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.630381 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:43.630566 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:43.630568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:43.630569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:43.630722 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:43.630813 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.630978 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.631068 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:37:43.634021 7275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:37:43.637004 7275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:43.651335 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.646401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.6.4.2:44655)
0927 18:37:43.646707 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.646718 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.646740 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:37:43.646811 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.646816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.646986 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.647136 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:43.647416 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.648313 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.649911 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.650611 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.650705 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.650870 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.651083 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.651176 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
I20260927 18:37:43.651328 7779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.646190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:44655 (local address 127.0.0.1:33938)
0927 18:37:43.646394 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.646403 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.646473 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.647128 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.647137 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.647172 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:43.647310 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.647317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.648405 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.648408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.649783 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.649794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.650738 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.650742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.650898 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.650938 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.651035 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.651180 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":50,"spinlock_wait_cycles":4736,"thread_start_us":67,"threads_started":1}
I20260927 18:37:43.651348 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.645417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39537 (local address 127.0.0.1:40508)
0927 18:37:43.646131 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.646143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.646232 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.647128 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.647134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.647169 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:43.647321 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.647328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.648684 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.648687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.649873 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.649881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.650742 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.650746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.650901 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.650939 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.651031 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.651189 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":597,"spinlock_wait_cycles":4736,"thread_start_us":88,"threads_started":1}
I20260927 18:37:43.651788 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.645415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:42599 (local address 127.0.0.1:43856)
0927 18:37:43.645798 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.645853 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.645981 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.647835 (+ 1854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.647838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.647856 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:43.647952 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.647958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.649179 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.649182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.650251 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.650262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.651450 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.651454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.651542 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.651561 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.651621 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.651678 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":19,"thread_start_us":101,"threads_started":1}
I20260927 18:37:43.652222 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.645546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.6.4.3:42599)
0927 18:37:43.647417 (+ 1871us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.647429 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.647449 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:37:43.647515 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.647520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.647685 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.647842 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:43.648064 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.649056 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.650386 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.651160 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.651247 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.651391 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.651957 (+ 566us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.652066 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1772,"thread_start_us":98,"threads_started":1}
I20260927 18:37:43.652275 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.645634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.6.4.1:39537)
0927 18:37:43.646612 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.646624 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.646647 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:37:43.646734 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.646739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.646980 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.647127 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:43.647451 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.648573 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.649958 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.650610 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.651734 (+ 1124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.652001 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.652046 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.652156 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":107,"threads_started":1}
I20260927 18:37:43.654326 7552 tablet_service.cc:1467] Processing CreateTablet for tablet fd8d15bcfc934071acf08d0c0266cb28 (DEFAULT_TABLE table=test-table [id=0c3d78a8c3c346db8ff3dd8f7bdf8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:43.655252 7552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd8d15bcfc934071acf08d0c0266cb28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:43.654881 7412 tablet_service.cc:1467] Processing CreateTablet for tablet fd8d15bcfc934071acf08d0c0266cb28 (DEFAULT_TABLE table=test-table [id=0c3d78a8c3c346db8ff3dd8f7bdf8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:43.654881 7692 tablet_service.cc:1467] Processing CreateTablet for tablet fd8d15bcfc934071acf08d0c0266cb28 (DEFAULT_TABLE table=test-table [id=0c3d78a8c3c346db8ff3dd8f7bdf8d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:43.655882 7412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd8d15bcfc934071acf08d0c0266cb28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:43.655891 7692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd8d15bcfc934071acf08d0c0266cb28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:43.661969 7781 tablet_bootstrap.cc:492] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Bootstrap starting.
I20260927 18:37:43.663467 7782 tablet_bootstrap.cc:492] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Bootstrap starting.
I20260927 18:37:43.664075 7781 tablet_bootstrap.cc:654] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:43.664400 7783 tablet_bootstrap.cc:492] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Bootstrap starting.
I20260927 18:37:43.664810 7781 log.cc:824] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:43.665380 7782 tablet_bootstrap.cc:654] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:43.666154 7782 log.cc:824] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:43.666286 7783 tablet_bootstrap.cc:654] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:43.666523 7781 tablet_bootstrap.cc:492] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: No bootstrap required, opened a new log
I20260927 18:37:43.666702 7781 ts_tablet_manager.cc:1397] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:37:43.667040 7783 log.cc:824] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:43.667954 7782 tablet_bootstrap.cc:492] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: No bootstrap required, opened a new log
I20260927 18:37:43.668146 7782 ts_tablet_manager.cc:1397] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:37:43.668871 7783 tablet_bootstrap.cc:492] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: No bootstrap required, opened a new log
I20260927 18:37:43.669044 7783 ts_tablet_manager.cc:1397] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:37:43.669761 7781 raft_consensus.cc:348] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.669958 7781 raft_consensus.cc:374] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:43.670007 7781 raft_consensus.cc:729] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2963a8c9e54b37a1246a3c492d837d, State: Initialized, Role: FOLLOWER
I20260927 18:37:43.670413 7781 consensus_queue.cc:260] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.671169 7781 ts_tablet_manager.cc:1428] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260927 18:37:43.671505 7618 heartbeater.cc:503] Master 127.6.4.62:33545 was elected leader, sending a full tablet report...
I20260927 18:37:43.672501 7782 raft_consensus.cc:348] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.672715 7782 raft_consensus.cc:374] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:43.672770 7782 raft_consensus.cc:729] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3397eb3abd4c4655a47fb0ec9d3f8571, State: Initialized, Role: FOLLOWER
I20260927 18:37:43.672646 7783 raft_consensus.cc:348] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.672839 7783 raft_consensus.cc:374] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:43.672873 7783 raft_consensus.cc:729] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f690855d58814e41b5bf0e2478e2baf7, State: Initialized, Role: FOLLOWER
I20260927 18:37:43.673136 7782 consensus_queue.cc:260] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.673194 7783 consensus_queue.cc:260] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.673738 7478 heartbeater.cc:503] Master 127.6.4.62:33545 was elected leader, sending a full tablet report...
I20260927 18:37:43.674044 7783 ts_tablet_manager.cc:1428] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:37:43.674046 7782 ts_tablet_manager.cc:1428] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260927 18:37:43.674309 7758 heartbeater.cc:503] Master 127.6.4.62:33545 was elected leader, sending a full tablet report...
W20260927 18:37:43.701390 7479 tablet.cc:2369] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:37:43.714879 7788 raft_consensus.cc:482] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:43.715034 7788 raft_consensus.cc:504] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.716080 7788 leader_election.cc:288] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599), af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655)
I20260927 18:37:43.720826 7776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.716411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:35067 (local address 127.6.4.3:42599)
0927 18:37:43.716532 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.716536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.716674 (+ 138us) server_negotiation.cc:407] Connection header received
0927 18:37:43.716795 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.716798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.716843 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.716919 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:43.717211 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.717996 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.719576 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.720212 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.720314 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.720371 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.720644 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.720711 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:37:43.720826 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.716555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:45347 (local address 127.6.4.2:44655)
0927 18:37:43.716671 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.716674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.716688 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:37:43.716795 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.716798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.716843 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.716919 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:43.717211 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.717997 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.719578 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.720218 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.720314 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.720371 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.720647 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.720711 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:37:43.720849 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.716303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:42599 (local address 127.6.4.1:35067)
0927 18:37:43.716578 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.716592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.716701 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.716935 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.716938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.716959 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:43.717129 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.717134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.718096 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.718099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.719455 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.719463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.720315 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.720319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.720511 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.720547 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.720624 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.720693 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":165,"spinlock_wait_cycles":10496,"thread_start_us":85,"threads_started":1}
I20260927 18:37:43.720986 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.716342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:44655 (local address 127.6.4.1:45347)
0927 18:37:43.716580 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:43.716602 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:43.716701 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:43.716935 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:43.716938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:43.716959 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:43.717113 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.717119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.718089 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.718092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.719454 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:43.719464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.720315 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.720319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:43.720506 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.720547 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:37:43.720625 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:37:43.720806 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":142,"spinlock_wait_cycles":30336,"thread_start_us":71,"threads_started":1}
I20260927 18:37:43.721477 7712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd8d15bcfc934071acf08d0c0266cb28" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f690855d58814e41b5bf0e2478e2baf7" is_pre_election: true
I20260927 18:37:43.721514 7572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd8d15bcfc934071acf08d0c0266cb28" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2963a8c9e54b37a1246a3c492d837d" is_pre_election: true
I20260927 18:37:43.721764 7712 raft_consensus.cc:2454] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 0.
I20260927 18:37:43.721764 7572 raft_consensus.cc:2454] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 0.
I20260927 18:37:43.722170 7365 leader_election.cc:302] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3397eb3abd4c4655a47fb0ec9d3f8571, af2963a8c9e54b37a1246a3c492d837d; no voters:
I20260927 18:37:43.722406 7788 raft_consensus.cc:2781] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:43.722479 7788 raft_consensus.cc:482] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:43.722530 7788 raft_consensus.cc:3037] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:43.723634 7788 raft_consensus.cc:504] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.723917 7788 leader_election.cc:288] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 election: Requested vote from peers f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599), af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655)
I20260927 18:37:43.724206 7572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd8d15bcfc934071acf08d0c0266cb28" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2963a8c9e54b37a1246a3c492d837d"
I20260927 18:37:43.724231 7712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd8d15bcfc934071acf08d0c0266cb28" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f690855d58814e41b5bf0e2478e2baf7"
I20260927 18:37:43.724309 7572 raft_consensus.cc:3037] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:43.724321 7712 raft_consensus.cc:3037] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:43.725237 7712 raft_consensus.cc:2454] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 1.
I20260927 18:37:43.725489 7364 leader_election.cc:302] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3397eb3abd4c4655a47fb0ec9d3f8571, f690855d58814e41b5bf0e2478e2baf7; no voters:
I20260927 18:37:43.725641 7572 raft_consensus.cc:2454] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 1.
I20260927 18:37:43.725659 7788 raft_consensus.cc:2781] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:43.725827 7788 raft_consensus.cc:686] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 LEADER]: Becoming Leader. State: Replica: 3397eb3abd4c4655a47fb0ec9d3f8571, State: Running, Role: LEADER
I20260927 18:37:43.726137 7788 consensus_queue.cc:237] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } }
I20260927 18:37:43.729024 7274 catalog_manager.cc:5495] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1). New cstate: current_term: 1 leader_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } health_report { overall_health: UNKNOWN } } }
I20260927 18:37:43.752223 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:43.748826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.6.4.1:39537)
0927 18:37:43.748942 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:43.748946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:43.748976 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:37:43.749185 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:43.749188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:43.749236 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:43.749313 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:43.749562 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.750037 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.750879 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:43.751084 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:43.751167 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:43.751468 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:43.751822 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:43.751914 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:43.752020 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 18:37:43.752072 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 18:37:43.757300 7572 raft_consensus.cc:1261] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Refusing update from remote peer 3397eb3abd4c4655a47fb0ec9d3f8571: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:43.757576 7712 raft_consensus.cc:1261] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Refusing update from remote peer 3397eb3abd4c4655a47fb0ec9d3f8571: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:43.757910 7788 consensus_queue.cc:1035] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:43.758090 7792 consensus_queue.cc:1035] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:43.765379 7796 mvcc.cc:205] Tried to move back new op lower bound from 7334028344343404544 to 7334028344224448512. Current Snapshot: MvccSnapshot[applied={T|T < 7334028344343404544}]
I20260927 18:37:43.768988 7797 mvcc.cc:205] Tried to move back new op lower bound from 7334028344343404544 to 7334028344224448512. Current Snapshot: MvccSnapshot[applied={T|T < 7334028344343404544}]
I20260927 18:37:43.773367 7274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:43.773545 7274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:43.775955 7274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: Sending DeleteTablet for 3 replicas of tablet fd8d15bcfc934071acf08d0c0266cb28
I20260927 18:37:43.776654 7692 tablet_service.cc:1514] Processing DeleteTablet for tablet fd8d15bcfc934071acf08d0c0266cb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:43 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:43856
I20260927 18:37:43.776742 7552 tablet_service.cc:1514] Processing DeleteTablet for tablet fd8d15bcfc934071acf08d0c0266cb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:43 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:33938
I20260927 18:37:43.777215 7412 tablet_service.cc:1514] Processing DeleteTablet for tablet fd8d15bcfc934071acf08d0c0266cb28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:43 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:40508
I20260927 18:37:43.777279 7804 tablet_replica.cc:330] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: stopping tablet replica
I20260927 18:37:43.777478 7274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:37:43.777616 7806 tablet_replica.cc:330] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: stopping tablet replica
W20260927 18:37:43.777849 7274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:43.777884 7806 raft_consensus.cc:2229] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:37:43.778152 7806 raft_consensus.cc:2258] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:43.779141 7806 ts_tablet_manager.cc:1905] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:43.779340 7804 raft_consensus.cc:2229] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:43.779611 7804 raft_consensus.cc:2258] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:43.781582 7806 ts_tablet_manager.cc:1918] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:37:43.781672 7806 log.cc:1196] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/fd8d15bcfc934071acf08d0c0266cb28
I20260927 18:37:43.781939 7806 ts_tablet_manager.cc:1939] T fd8d15bcfc934071acf08d0c0266cb28 P 3397eb3abd4c4655a47fb0ec9d3f8571: Deleting consensus metadata
I20260927 18:37:43.782728 7552 tablet_service.cc:1467] Processing CreateTablet for tablet 0eea948f1f4c46d98f3274b0d7099e2b (DEFAULT_TABLE table=test-table [id=02a657acac7149de87c2ed1028f4a72d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:43.782743 7692 tablet_service.cc:1467] Processing CreateTablet for tablet 0eea948f1f4c46d98f3274b0d7099e2b (DEFAULT_TABLE table=test-table [id=02a657acac7149de87c2ed1028f4a72d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:43.782961 7692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eea948f1f4c46d98f3274b0d7099e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:43.782961 7552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eea948f1f4c46d98f3274b0d7099e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:43.784713 7783 tablet_bootstrap.cc:492] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Bootstrap starting.
I20260927 18:37:43.785214 7261 catalog_manager.cc:4841] TS 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1:39537): tablet fd8d15bcfc934071acf08d0c0266cb28 (table test-table [id=0c3d78a8c3c346db8ff3dd8f7bdf8d64]) successfully deleted
I20260927 18:37:43.785589 7783 tablet_bootstrap.cc:654] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:43.785632 7412 tablet_service.cc:1467] Processing CreateTablet for tablet 0eea948f1f4c46d98f3274b0d7099e2b (DEFAULT_TABLE table=test-table [id=02a657acac7149de87c2ed1028f4a72d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:43.785848 7412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eea948f1f4c46d98f3274b0d7099e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:43.786566 7783 tablet_bootstrap.cc:492] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: No bootstrap required, opened a new log
I20260927 18:37:43.786641 7783 ts_tablet_manager.cc:1397] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:43.787010 7783 raft_consensus.cc:348] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.787102 7783 raft_consensus.cc:374] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:43.787139 7783 raft_consensus.cc:729] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f690855d58814e41b5bf0e2478e2baf7, State: Initialized, Role: FOLLOWER
I20260927 18:37:43.787249 7783 consensus_queue.cc:260] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.787467 7783 ts_tablet_manager.cc:1428] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:43.787806 7782 tablet_bootstrap.cc:492] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Bootstrap starting.
I20260927 18:37:43.788481 7781 tablet_bootstrap.cc:492] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Bootstrap starting.
I20260927 18:37:43.788566 7782 tablet_bootstrap.cc:654] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:43.789142 7804 ts_tablet_manager.cc:1905] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:43.789358 7781 tablet_bootstrap.cc:654] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:43.790757 7781 tablet_bootstrap.cc:492] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: No bootstrap required, opened a new log
I20260927 18:37:43.790833 7781 ts_tablet_manager.cc:1397] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:43.791065 7782 tablet_bootstrap.cc:492] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: No bootstrap required, opened a new log
I20260927 18:37:43.791131 7782 ts_tablet_manager.cc:1397] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:37:43.791258 7781 raft_consensus.cc:348] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.791342 7781 raft_consensus.cc:374] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:43.791371 7781 raft_consensus.cc:729] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2963a8c9e54b37a1246a3c492d837d, State: Initialized, Role: FOLLOWER
I20260927 18:37:43.791471 7804 ts_tablet_manager.cc:1918] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:37:43.791472 7781 consensus_queue.cc:260] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.791496 7782 raft_consensus.cc:348] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.791546 7804 log.cc:1196] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/fd8d15bcfc934071acf08d0c0266cb28
I20260927 18:37:43.791579 7782 raft_consensus.cc:374] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:43.791620 7782 raft_consensus.cc:729] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3397eb3abd4c4655a47fb0ec9d3f8571, State: Initialized, Role: FOLLOWER
I20260927 18:37:43.791705 7781 ts_tablet_manager.cc:1428] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:43.791723 7782 consensus_queue.cc:260] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.791941 7782 ts_tablet_manager.cc:1428] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:43.792239 7805 tablet_replica.cc:330] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: stopping tablet replica
I20260927 18:37:43.792570 7805 raft_consensus.cc:2229] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:43.792869 7805 raft_consensus.cc:2258] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:43.793756 7804 ts_tablet_manager.cc:1939] T fd8d15bcfc934071acf08d0c0266cb28 P f690855d58814e41b5bf0e2478e2baf7: Deleting consensus metadata
I20260927 18:37:43.794314 7805 ts_tablet_manager.cc:1905] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:43.794909 7260 catalog_manager.cc:4841] TS f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599): tablet fd8d15bcfc934071acf08d0c0266cb28 (table test-table [id=0c3d78a8c3c346db8ff3dd8f7bdf8d64]) successfully deleted
I20260927 18:37:43.796720 7805 ts_tablet_manager.cc:1918] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:37:43.796787 7805 log.cc:1196] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/fd8d15bcfc934071acf08d0c0266cb28
I20260927 18:37:43.796995 7805 ts_tablet_manager.cc:1939] T fd8d15bcfc934071acf08d0c0266cb28 P af2963a8c9e54b37a1246a3c492d837d: Deleting consensus metadata
I20260927 18:37:43.797865 7261 catalog_manager.cc:4841] TS af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655): tablet fd8d15bcfc934071acf08d0c0266cb28 (table test-table [id=0c3d78a8c3c346db8ff3dd8f7bdf8d64]) successfully deleted
W20260927 18:37:43.847833 7759 tablet.cc:2369] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:37:43.904387 7619 tablet.cc:2369] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:37:43.966543 7792 raft_consensus.cc:482] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:43.966697 7792 raft_consensus.cc:504] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.967113 7792 leader_election.cc:288] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655), f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599)
I20260927 18:37:43.967545 7572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea948f1f4c46d98f3274b0d7099e2b" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2963a8c9e54b37a1246a3c492d837d" is_pre_election: true
I20260927 18:37:43.967589 7712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea948f1f4c46d98f3274b0d7099e2b" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f690855d58814e41b5bf0e2478e2baf7" is_pre_election: true
I20260927 18:37:43.967680 7572 raft_consensus.cc:2454] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 0.
I20260927 18:37:43.967744 7712 raft_consensus.cc:2454] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 0.
I20260927 18:37:43.967958 7365 leader_election.cc:302] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3397eb3abd4c4655a47fb0ec9d3f8571, af2963a8c9e54b37a1246a3c492d837d; no voters:
I20260927 18:37:43.968127 7792 raft_consensus.cc:2781] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:43.968182 7792 raft_consensus.cc:482] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:43.968209 7792 raft_consensus.cc:3037] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:43.969210 7792 raft_consensus.cc:504] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.969506 7792 leader_election.cc:288] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 election: Requested vote from peers af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655), f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599)
I20260927 18:37:43.969841 7712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea948f1f4c46d98f3274b0d7099e2b" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f690855d58814e41b5bf0e2478e2baf7"
I20260927 18:37:43.969861 7572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eea948f1f4c46d98f3274b0d7099e2b" candidate_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af2963a8c9e54b37a1246a3c492d837d"
I20260927 18:37:43.969934 7712 raft_consensus.cc:3037] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:43.969942 7572 raft_consensus.cc:3037] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:43.970745 7572 raft_consensus.cc:2454] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 1.
I20260927 18:37:43.970852 7712 raft_consensus.cc:2454] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3397eb3abd4c4655a47fb0ec9d3f8571 in term 1.
I20260927 18:37:43.970996 7365 leader_election.cc:302] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3397eb3abd4c4655a47fb0ec9d3f8571, af2963a8c9e54b37a1246a3c492d837d; no voters:
I20260927 18:37:43.971135 7792 raft_consensus.cc:2781] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:43.971215 7792 raft_consensus.cc:686] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 LEADER]: Becoming Leader. State: Replica: 3397eb3abd4c4655a47fb0ec9d3f8571, State: Running, Role: LEADER
I20260927 18:37:43.971390 7792 consensus_queue.cc:237] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:43.972973 7275 catalog_manager.cc:5495] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1). New cstate: current_term: 1 leader_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } health_report { overall_health: UNKNOWN } } }
Sep 27 18:37:44 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534262, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:44.045375 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.037798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33545 (local address 127.0.0.1:34032)
0927 18:37:44.038050 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:44.038061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:44.038168 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:44.038392 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:44.038394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:44.038773 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:44.038870 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.038876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.039583 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.039586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.040312 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.040321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.040702 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.040714 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.040772 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.041370 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:44.041382 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:44.042671 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:44.043994 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:44.044002 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:44.044007 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:44.044276 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:44.044471 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:44.044473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:44.044474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:44.044546 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:44.044859 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:44.044864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:44.044978 (+ 114us) client_negotiation.cc:769] Sending connection context
0927 18:37:44.045098 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 18:37:44.045216 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260927 18:37:44.045388 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.037839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.6.4.62:33545)
0927 18:37:44.037957 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.037961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.038129 (+ 168us) server_negotiation.cc:407] Connection header received
0927 18:37:44.038237 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.038239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.038300 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.038373 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:44.038962 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.039464 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.040419 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.040625 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.040696 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.042787 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:44.042790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:44.042792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:44.042812 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:44.043900 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:44.044343 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:44.044345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:44.044347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:44.044409 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:44.044596 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:44.044599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:44.044601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:44.044746 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:44.044813 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.045149 (+ 336us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.045266 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":42}
I20260927 18:37:44.049022 7275 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:34032:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:37:44.049728 7275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:44.054152 7412 tablet_service.cc:1467] Processing CreateTablet for tablet ee34dc836aaa44cc82b0e962e7b7025e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04f951bd590a422a842a270a4741ebfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:37:44.054244 7552 tablet_service.cc:1467] Processing CreateTablet for tablet ee34dc836aaa44cc82b0e962e7b7025e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04f951bd590a422a842a270a4741ebfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:37:44.054268 7692 tablet_service.cc:1467] Processing CreateTablet for tablet ee34dc836aaa44cc82b0e962e7b7025e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=04f951bd590a422a842a270a4741ebfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:37:44.054543 7412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee34dc836aaa44cc82b0e962e7b7025e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:44.054582 7692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee34dc836aaa44cc82b0e962e7b7025e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:44.054582 7552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee34dc836aaa44cc82b0e962e7b7025e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:44.056876 7782 tablet_bootstrap.cc:492] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: Bootstrap starting.
I20260927 18:37:44.057286 7781 tablet_bootstrap.cc:492] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: Bootstrap starting.
I20260927 18:37:44.057657 7782 tablet_bootstrap.cc:654] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:44.057669 7783 tablet_bootstrap.cc:492] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: Bootstrap starting.
I20260927 18:37:44.058123 7781 tablet_bootstrap.cc:654] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:44.058467 7783 tablet_bootstrap.cc:654] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:44.058763 7782 tablet_bootstrap.cc:492] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: No bootstrap required, opened a new log
I20260927 18:37:44.058826 7782 ts_tablet_manager.cc:1397] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:44.059161 7782 raft_consensus.cc:348] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.059284 7782 raft_consensus.cc:374] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:44.059320 7782 raft_consensus.cc:729] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3397eb3abd4c4655a47fb0ec9d3f8571, State: Initialized, Role: FOLLOWER
I20260927 18:37:44.059466 7783 tablet_bootstrap.cc:492] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: No bootstrap required, opened a new log
I20260927 18:37:44.059466 7781 tablet_bootstrap.cc:492] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: No bootstrap required, opened a new log
I20260927 18:37:44.059429 7782 consensus_queue.cc:260] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.059533 7783 ts_tablet_manager.cc:1397] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:44.059533 7781 ts_tablet_manager.cc:1397] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:44.059700 7782 ts_tablet_manager.cc:1428] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:44.059901 7792 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.059926 7781 raft_consensus.cc:348] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.059927 7783 raft_consensus.cc:348] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.060012 7792 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.060024 7781 raft_consensus.cc:374] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:44.060029 7783 raft_consensus.cc:374] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:44.060058 7783 raft_consensus.cc:729] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f690855d58814e41b5bf0e2478e2baf7, State: Initialized, Role: FOLLOWER
I20260927 18:37:44.060056 7781 raft_consensus.cc:729] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2963a8c9e54b37a1246a3c492d837d, State: Initialized, Role: FOLLOWER
I20260927 18:37:44.060191 7783 consensus_queue.cc:260] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.060191 7781 consensus_queue.cc:260] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.060439 7783 ts_tablet_manager.cc:1428] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:37:44.060439 7781 ts_tablet_manager.cc:1428] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:44.060619 7789 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.060676 7787 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.060752 7789 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.060771 7787 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.301435 7787 raft_consensus.cc:482] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:44.301569 7787 raft_consensus.cc:504] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.302655 7787 leader_election.cc:288] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1:39537), f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599)
I20260927 18:37:44.306509 7822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.302997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:48775 (local address 127.6.4.1:39537)
0927 18:37:44.303249 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.303253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.303298 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:37:44.303469 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.303472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.303512 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.303582 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:44.303818 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.304401 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.305535 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.305989 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.306072 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.306138 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.306312 (+ 174us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.306382 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":55,"threads_started":1}
I20260927 18:37:44.306519 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.302897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:42599 (local address 127.6.4.2:52409)
0927 18:37:44.303348 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:44.303359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:44.303459 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:44.303693 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:44.303695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:44.303709 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:44.303810 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.303814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.304565 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.304568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.305432 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.305441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.306119 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.306123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.306214 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.306231 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:37:44.306297 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:37:44.306364 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":224,"thread_start_us":71,"threads_started":1}
I20260927 18:37:44.306509 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.302979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:52409 (local address 127.6.4.3:42599)
0927 18:37:44.303226 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.303231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.303437 (+ 206us) server_negotiation.cc:407] Connection header received
0927 18:37:44.303556 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.303559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.303600 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.303667 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:44.303908 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.304468 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.305591 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.306010 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.306117 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.306177 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.306320 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.306391 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260927 18:37:44.306520 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.302891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39537 (local address 127.6.4.2:48775)
0927 18:37:44.303226 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:44.303240 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:44.303363 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:44.303608 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:44.303611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:44.303626 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:44.303721 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.303726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.304506 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.304510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.305412 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.305421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.306102 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.306105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.306193 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.306216 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:37:44.306281 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:37:44.306354 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":21,"thread_start_us":78,"threads_started":1}
I20260927 18:37:44.306923 7712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee34dc836aaa44cc82b0e962e7b7025e" candidate_uuid: "af2963a8c9e54b37a1246a3c492d837d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f690855d58814e41b5bf0e2478e2baf7" is_pre_election: true
I20260927 18:37:44.307057 7712 raft_consensus.cc:2454] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2963a8c9e54b37a1246a3c492d837d in term 0.
I20260927 18:37:44.307399 7504 leader_election.cc:302] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af2963a8c9e54b37a1246a3c492d837d, f690855d58814e41b5bf0e2478e2baf7; no voters:
I20260927 18:37:44.307395 7432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee34dc836aaa44cc82b0e962e7b7025e" candidate_uuid: "af2963a8c9e54b37a1246a3c492d837d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" is_pre_election: true
I20260927 18:37:44.307619 7432 raft_consensus.cc:2454] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af2963a8c9e54b37a1246a3c492d837d in term 0.
I20260927 18:37:44.307680 7787 raft_consensus.cc:2781] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:44.307731 7787 raft_consensus.cc:482] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:44.307760 7787 raft_consensus.cc:3037] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:44.308804 7787 raft_consensus.cc:504] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.309131 7787 leader_election.cc:288] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [CANDIDATE]: Term 1 election: Requested vote from peers 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1:39537), f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599)
I20260927 18:37:44.309419 7432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee34dc836aaa44cc82b0e962e7b7025e" candidate_uuid: "af2963a8c9e54b37a1246a3c492d837d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571"
I20260927 18:37:44.309453 7712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee34dc836aaa44cc82b0e962e7b7025e" candidate_uuid: "af2963a8c9e54b37a1246a3c492d837d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f690855d58814e41b5bf0e2478e2baf7"
I20260927 18:37:44.309530 7432 raft_consensus.cc:3037] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:44.309543 7712 raft_consensus.cc:3037] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:44.310552 7432 raft_consensus.cc:2454] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2963a8c9e54b37a1246a3c492d837d in term 1.
I20260927 18:37:44.310552 7712 raft_consensus.cc:2454] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af2963a8c9e54b37a1246a3c492d837d in term 1.
I20260927 18:37:44.310819 7505 leader_election.cc:302] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3397eb3abd4c4655a47fb0ec9d3f8571, af2963a8c9e54b37a1246a3c492d837d; no voters:
I20260927 18:37:44.311021 7787 raft_consensus.cc:2781] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:44.311167 7787 raft_consensus.cc:686] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [term 1 LEADER]: Becoming Leader. State: Replica: af2963a8c9e54b37a1246a3c492d837d, State: Running, Role: LEADER
I20260927 18:37:44.311385 7787 consensus_queue.cc:237] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } }
I20260927 18:37:44.312497 7824 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: TxnStatusTablet state changed. Reason: New leader af2963a8c9e54b37a1246a3c492d837d. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.312613 7824 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:37:44.312849 7826 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:37:44.312994 7826 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:37:44.313777 7275 catalog_manager.cc:5495] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d reported cstate change: term changed from 0 to 1, leader changed from <none> to af2963a8c9e54b37a1246a3c492d837d (127.6.4.2). New cstate: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } health_report { overall_health: UNKNOWN } } }
I20260927 18:37:44.351469 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.347474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:44655 (local address 127.0.0.1:33946)
0927 18:37:44.347721 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:44.347737 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:44.347837 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:44.348184 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:44.348187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:44.348198 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:37:44.348308 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.348314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.349107 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.349111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.349827 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.349838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.350281 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.350297 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.350388 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.350435 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:37:44.351176 (+ 741us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:37:44.351192 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:37:44.351255 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:37:44.351344 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260927 18:37:44.351486 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.347594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.6.4.2:44655)
0927 18:37:44.347842 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.347849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.347868 (+ 19us) server_negotiation.cc:407] Connection header received
0927 18:37:44.347968 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.347970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.348023 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.348168 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:44.348403 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.348984 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.349951 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.350188 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.350284 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.350579 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:44.351081 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:44.351187 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.351266 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.351317 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
I20260927 18:37:44.354699 7712 raft_consensus.cc:1261] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Refusing update from remote peer af2963a8c9e54b37a1246a3c492d837d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:44.354876 7432 raft_consensus.cc:1261] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Refusing update from remote peer af2963a8c9e54b37a1246a3c492d837d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:44.355216 7787 consensus_queue.cc:1035] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:44.355387 7824 consensus_queue.cc:1035] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:44.356815 7789 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: TxnStatusTablet state changed. Reason: New leader af2963a8c9e54b37a1246a3c492d837d. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.356927 7789 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.356956 7792 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: TxnStatusTablet state changed. Reason: New leader af2963a8c9e54b37a1246a3c492d837d. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.357064 7792 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.359298 7824 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.359396 7824 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:37:44.359570 7789 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.359658 7789 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P f690855d58814e41b5bf0e2478e2baf7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.359338 7787 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.360034 7787 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P af2963a8c9e54b37a1246a3c492d837d: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:37:44.360603 7832 mvcc.cc:205] Tried to move back new op lower bound from 7334028346791964672 to 7334028346621407232. Current Snapshot: MvccSnapshot[applied={T|T < 7334028346791964672}]
I20260927 18:37:44.360762 7792 tablet_replica.cc:439] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af2963a8c9e54b37a1246a3c492d837d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3397eb3abd4c4655a47fb0ec9d3f8571" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39537 } } peers { permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 } } peers { permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 } } }
I20260927 18:37:44.360845 7792 tablet_replica.cc:442] T ee34dc836aaa44cc82b0e962e7b7025e P 3397eb3abd4c4655a47fb0ec9d3f8571: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:44.379758 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.376610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:55427 (local address 127.6.4.2:44655)
0927 18:37:44.376729 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.376732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.376745 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:37:44.376835 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.376837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.376874 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.376932 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:44.377168 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.377650 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.378848 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.379065 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.379136 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.379415 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:44.379475 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:44.379522 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.379617 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.379658 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:37:44.379841 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.376347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:44655 (local address 127.6.4.1:55427)
0927 18:37:44.376649 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:44.376665 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:44.376757 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:44.376962 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:44.376965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:44.376972 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:37:44.377075 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.377080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.377750 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.377753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.378733 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.378743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.379155 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.379170 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.379258 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.379306 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:37:44.379545 (+ 239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:37:44.379550 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:37:44.379608 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:37:44.379691 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":99,"threads_started":1}
I20260927 18:37:44.386035 7712 raft_consensus.cc:1261] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Refusing update from remote peer 3397eb3abd4c4655a47fb0ec9d3f8571: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:44.386466 7788 consensus_queue.cc:1035] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f690855d58814e41b5bf0e2478e2baf7" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:44.386533 7572 raft_consensus.cc:1261] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Refusing update from remote peer 3397eb3abd4c4655a47fb0ec9d3f8571: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:44.386876 7788 consensus_queue.cc:1035] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2963a8c9e54b37a1246a3c492d837d" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:44.406282 7822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.401866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:37955 (local address 127.6.4.1:39537)
0927 18:37:44.401961 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.401965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.402022 (+ 57us) server_negotiation.cc:407] Connection header received
0927 18:37:44.402148 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.402150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.402187 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.402243 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:44.402493 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.403225 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.405083 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.405427 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.405525 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.405828 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:44.405911 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:44.405953 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.406109 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.406162 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 18:37:44.406311 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.401550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39537 (local address 127.6.4.2:37955)
0927 18:37:44.401949 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:44.401963 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:44.402060 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:44.402264 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:44.402267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:44.402273 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:37:44.402402 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.402408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.403344 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.403348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.404961 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:44.404973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.405534 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.405549 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:44.405633 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.405681 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:37:44.406010 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:37:44.406015 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:37:44.406074 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:37:44.406159 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":92,"threads_started":1}
I20260927 18:37:44.428757 7275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:44.428920 7275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:44.431012 7275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a2db856edd49e09711882a061d813f: Sending DeleteTablet for 3 replicas of tablet 0eea948f1f4c46d98f3274b0d7099e2b
I20260927 18:37:44.431542 7552 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eea948f1f4c46d98f3274b0d7099e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:44 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:33938
I20260927 18:37:44.431595 7692 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eea948f1f4c46d98f3274b0d7099e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:44 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:43856
I20260927 18:37:44.431911 7848 tablet_replica.cc:330] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: stopping tablet replica
I20260927 18:37:44.432058 7848 raft_consensus.cc:2229] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:44.432070 7412 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eea948f1f4c46d98f3274b0d7099e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:44 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:40508
I20260927 18:37:44.432199 7848 raft_consensus.cc:2258] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:44.432785 7848 ts_tablet_manager.cc:1905] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:44.433032 7849 tablet_replica.cc:330] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: stopping tablet replica
I20260927 18:37:44.433363 7849 raft_consensus.cc:2229] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:37:44.433574 7849 raft_consensus.cc:2258] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:44.434141 7849 ts_tablet_manager.cc:1905] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:44.435160 7848 ts_tablet_manager.cc:1918] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:44.435225 7848 log.cc:1196] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/0eea948f1f4c46d98f3274b0d7099e2b
I20260927 18:37:44.435425 7848 ts_tablet_manager.cc:1939] T 0eea948f1f4c46d98f3274b0d7099e2b P f690855d58814e41b5bf0e2478e2baf7: Deleting consensus metadata
I20260927 18:37:44.436085 7260 catalog_manager.cc:4841] TS f690855d58814e41b5bf0e2478e2baf7 (127.6.4.3:42599): tablet 0eea948f1f4c46d98f3274b0d7099e2b (table test-table [id=02a657acac7149de87c2ed1028f4a72d]) successfully deleted
I20260927 18:37:44.437029 7849 ts_tablet_manager.cc:1918] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:44.437119 7849 log.cc:1196] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/0eea948f1f4c46d98f3274b0d7099e2b
I20260927 18:37:44.437435 7849 ts_tablet_manager.cc:1939] T 0eea948f1f4c46d98f3274b0d7099e2b P 3397eb3abd4c4655a47fb0ec9d3f8571: Deleting consensus metadata
I20260927 18:37:44.438064 7261 catalog_manager.cc:4841] TS 3397eb3abd4c4655a47fb0ec9d3f8571 (127.6.4.1:39537): tablet 0eea948f1f4c46d98f3274b0d7099e2b (table test-table [id=02a657acac7149de87c2ed1028f4a72d]) successfully deleted
I20260927 18:37:44.440341 7847 tablet_replica.cc:330] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: stopping tablet replica
I20260927 18:37:44.440481 7847 raft_consensus.cc:2229] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:44.440577 7847 raft_consensus.cc:2258] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 27 18:37:44 dist-test-slave-bzzj krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:44.441246 7847 ts_tablet_manager.cc:1905] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:44.443892 7847 ts_tablet_manager.cc:1918] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:44.443979 7847 log.cc:1196] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/0eea948f1f4c46d98f3274b0d7099e2b
I20260927 18:37:44.444242 7847 ts_tablet_manager.cc:1939] T 0eea948f1f4c46d98f3274b0d7099e2b P af2963a8c9e54b37a1246a3c492d837d: Deleting consensus metadata
I20260927 18:37:44.444921 7261 catalog_manager.cc:4841] TS af2963a8c9e54b37a1246a3c492d837d (127.6.4.2:44655): tablet 0eea948f1f4c46d98f3274b0d7099e2b (table test-table [id=02a657acac7149de87c2ed1028f4a72d]) successfully deleted
I20260927 18:37:44.445123 7822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.433160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.6.4.1:39537)
0927 18:37:44.433273 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.433277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.435744 (+ 2467us) server_negotiation.cc:407] Connection header received
0927 18:37:44.435921 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.435925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.435971 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.436021 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:44.436859 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.437491 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.438390 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.438603 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.438691 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.441190 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:44.441198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:44.441212 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:44.441236 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:44.443127 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:44.443585 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:44.443591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:44.443593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:44.443695 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:44.444008 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:44.444011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:44.444013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:44.444333 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:44.444500 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.444859 (+ 359us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.444967 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260927 18:37:44.445478 7472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40524
I20260927 18:37:44.453999 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:44.447094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.6.4.62:33545)
0927 18:37:44.447238 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:44.447243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:44.447258 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:37:44.447345 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:44.447348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:44.447395 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:44.447477 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:44.447981 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.448775 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.449514 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:44.449842 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:44.449953 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:44.451079 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:44.451083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:44.451086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:44.451119 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:44.452525 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:44.452959 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:44.452962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:44.452964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:44.453041 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:44.453246 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:44.453248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:44.453250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:44.453429 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:44.453560 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:44.453743 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 18:37:44.453844 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260927 18:37:44.454330 7275 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34044
I20260927 18:37:44.455348 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7347
I20260927 18:37:44.463932 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7487
I20260927 18:37:44.469345 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7627
I20260927 18:37:44.473721 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7243
2026-09-27T18:37:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2774 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:44 dist-test-slave-bzzj krb5kdc[7856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:44 dist-test-slave-bzzj krb5kdc[7856](info): set up 2 sockets
Sep 27 18:37:44 dist-test-slave-bzzj krb5kdc[7856](info): commencing operation
krb5kdc: starting...
W20260927 18:37:45.431684 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:45 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:45.538954 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:36259
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:44495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:36259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:45.642963 7869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:45.643214 7869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:45.643265 7869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:45.647248 7869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:45.647300 7869 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:37:45.647321 7869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:45.647344 7869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:45.647367 7869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:45.647393 7869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:45.652395 7869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:44495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:36259
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:36259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:45.653308 7869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:45.654258 7869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:45.662143 7874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:45.662174 7877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:45.662281 7869 server_base.cc:1034] running on GCE node
W20260927 18:37:45.663062 7875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:45.663589 7869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:45.664620 7869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:45.665781 7869 hybrid_clock.cc:648] HybridClock initialized: now 1790534265665764 us; error 29 us; skew 500 ppm
Sep 27 18:37:45 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534265, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:45.668335 7869 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:45.669090 7869 webserver.cc:460] Webserver started at http://127.6.4.62:40367/ using document root <none> and password file <none>
I20260927 18:37:45.669553 7869 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:45.669613 7869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:45.669765 7869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:45.671603 7869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "c24d78ebda514fb1b7a8310e47ecea9a"
format_stamp: "Formatted at 2026-09-27 18:37:45 on dist-test-slave-bzzj"
server_key: "cf6dbe9a28abab36ff7f17b161decf64"
server_key_iv: "6575fc70d33582fda6b0c2b0abc304d7"
server_key_version: "encryptionkey@0"
I20260927 18:37:45.672080 7869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "c24d78ebda514fb1b7a8310e47ecea9a"
format_stamp: "Formatted at 2026-09-27 18:37:45 on dist-test-slave-bzzj"
server_key: "cf6dbe9a28abab36ff7f17b161decf64"
server_key_iv: "6575fc70d33582fda6b0c2b0abc304d7"
server_key_version: "encryptionkey@0"
I20260927 18:37:45.675419 7869 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:37:45.677596 7884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:45.678727 7869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:45.678826 7869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "c24d78ebda514fb1b7a8310e47ecea9a"
format_stamp: "Formatted at 2026-09-27 18:37:45 on dist-test-slave-bzzj"
server_key: "cf6dbe9a28abab36ff7f17b161decf64"
server_key_iv: "6575fc70d33582fda6b0c2b0abc304d7"
server_key_version: "encryptionkey@0"
I20260927 18:37:45.678918 7869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:45.724277 7869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:45.727301 7869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:45.727473 7869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:45.734611 7869 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:36259
I20260927 18:37:45.734633 7946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:36259 every 8 connection(s)
I20260927 18:37:45.735459 7869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:45.737990 7947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:45.743038 7947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Bootstrap starting.
I20260927 18:37:45.744560 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7869
I20260927 18:37:45.744671 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:45.744935 6160 external_mini_cluster.cc:1428] Setting key e54794b00281811cd5553d9b4bf4e54e
I20260927 18:37:45.745529 7947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:45.746358 7947 log.cc:824] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:45.748040 7947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: No bootstrap required, opened a new log
I20260927 18:37:45.751220 7947 raft_consensus.cc:348] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36259 } }
I20260927 18:37:45.751427 7947 raft_consensus.cc:374] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:45.751488 7947 raft_consensus.cc:729] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c24d78ebda514fb1b7a8310e47ecea9a, State: Initialized, Role: FOLLOWER
I20260927 18:37:45.751884 7947 consensus_queue.cc:260] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36259 } }
I20260927 18:37:45.752012 7947 raft_consensus.cc:388] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:45.752074 7947 raft_consensus.cc:482] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:45.752192 7947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:45.752933 7947 raft_consensus.cc:504] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36259 } }
I20260927 18:37:45.753229 7947 leader_election.cc:302] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c24d78ebda514fb1b7a8310e47ecea9a; no voters:
I20260927 18:37:45.753473 7947 leader_election.cc:288] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:45.753609 7952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:45.753845 7952 raft_consensus.cc:686] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [term 1 LEADER]: Becoming Leader. State: Replica: c24d78ebda514fb1b7a8310e47ecea9a, State: Running, Role: LEADER
I20260927 18:37:45.754107 7952 consensus_queue.cc:237] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36259 } }
I20260927 18:37:45.754583 7947 sys_catalog.cc:564] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [sys.catalog]: configured and running, proceeding with master startup.
Sep 27 18:37:45 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:45.756896 7954 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c24d78ebda514fb1b7a8310e47ecea9a. Latest consensus state: current_term: 1 leader_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36259 } } }
I20260927 18:37:45.757000 7954 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:45.757218 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d78ebda514fb1b7a8310e47ecea9a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36259 } } }
I20260927 18:37:45.757314 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a [sys.catalog]: This master's current role is: LEADER
W20260927 18:37:45.760272 7966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:37:45.760339 7966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:37:45.760406 7967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:45.761214 7967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:45.761986 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:45.745970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.6.4.62:36259)
0927 18:37:45.746359 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:45.746372 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:45.746405 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:37:45.747141 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:45.747160 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:45.747458 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:45.747750 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:45.748514 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:45.749503 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:45.750335 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:45.750801 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:45.750949 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:45.756311 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:45.756317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:45.756334 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:45.756367 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:45.758791 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:45.759278 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:45.759284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:45.759289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:45.759419 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:45.759685 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:45.759688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:45.759690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:45.760183 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:45.760366 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:45.760667 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 18:37:45.761102 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260927 18:37:45.766212 7967 catalog_manager.cc:1347] Generated new cluster ID: 0b172758017f4912aeb22ce69f6c71a0
I20260927 18:37:45.766268 7967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:45.792785 7967 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:45.793723 7967 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:45.801733 7967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Generated new TSK 0
I20260927 18:37:45.802291 7967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:45.824738 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36259
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:44495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:45.927254 7973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:45.927464 7973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:45.927505 7973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:45.931404 7973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:45.931456 7973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:45.931492 7973 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:37:45.931504 7973 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:37:45.931533 7973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:45.936414 7973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:44495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:36259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:45.937234 7973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:45.938334 7973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:45.946844 7979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:45.946856 7981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:45.946882 7978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:45.947180 7973 server_base.cc:1034] running on GCE node
I20260927 18:37:45.947510 7973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:45.948048 7973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:45.949230 7973 hybrid_clock.cc:648] HybridClock initialized: now 1790534265949215 us; error 27 us; skew 500 ppm
Sep 27 18:37:45 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534265, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:45.951604 7973 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:45.952383 7973 webserver.cc:460] Webserver started at http://127.6.4.1:35837/ using document root <none> and password file <none>
I20260927 18:37:45.952865 7973 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:45.952926 7973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:45.953097 7973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:45.954988 7973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "eb4fc38a6b2748699758376ce65801bc"
format_stamp: "Formatted at 2026-09-27 18:37:45 on dist-test-slave-bzzj"
server_key: "23ff041e0104d729e3b816774164d0b1"
server_key_iv: "bf224e8100125476bf1ba495f0b63ad5"
server_key_version: "encryptionkey@0"
I20260927 18:37:45.955498 7973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "eb4fc38a6b2748699758376ce65801bc"
format_stamp: "Formatted at 2026-09-27 18:37:45 on dist-test-slave-bzzj"
server_key: "23ff041e0104d729e3b816774164d0b1"
server_key_iv: "bf224e8100125476bf1ba495f0b63ad5"
server_key_version: "encryptionkey@0"
I20260927 18:37:45.958768 7973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260927 18:37:45.960924 7988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:45.961994 7973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:45.962096 7973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "eb4fc38a6b2748699758376ce65801bc"
format_stamp: "Formatted at 2026-09-27 18:37:45 on dist-test-slave-bzzj"
server_key: "23ff041e0104d729e3b816774164d0b1"
server_key_iv: "bf224e8100125476bf1ba495f0b63ad5"
server_key_version: "encryptionkey@0"
I20260927 18:37:45.962169 7973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:45.984366 7973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:45.986794 7973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:45.986924 7973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:45.988191 7973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:45.988237 7973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:45.988276 7973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:45.988301 7973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:37:45 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534265, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:45.997618 7973 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:39271
I20260927 18:37:45.997673 8103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:39271 every 8 connection(s)
I20260927 18:37:45.998466 7973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:37:45.999302 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 7973
I20260927 18:37:45.999418 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:45.999718 6160 external_mini_cluster.cc:1428] Setting key 09d52e342b2efd03c9923c5d6b4efa9b
I20260927 18:37:46.000429 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:45.988822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:34841 (local address 127.6.4.62:36259)
0927 18:37:45.988958 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:45.988964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:45.989794 (+ 830us) server_negotiation.cc:407] Connection header received
0927 18:37:45.990741 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:45.990745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:45.990797 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:45.990850 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:45.992338 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:45.992998 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:45.994163 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:45.994460 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:45.994548 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:45.997045 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:45.997048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:45.997050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:45.997073 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:45.998384 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:45.998938 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:45.998941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:45.998943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:45.999034 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:45.999367 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:45.999371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:45.999373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:45.999583 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:45.999693 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.000154 (+ 461us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.000277 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:37:46.001160 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:45.989231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.6.4.1:34841)
0927 18:37:45.989651 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:45.989696 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:45.990512 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:45.991242 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:45.991251 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:45.991602 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:45.992206 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:45.992221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:45.993101 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:45.993105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:45.994058 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:45.994067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:45.994556 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:45.994606 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:45.994783 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:45.995267 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:45.995282 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:45.996895 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:45.998493 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:45.998499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:45.998512 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:45.998825 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:45.999134 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:45.999137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:45.999139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:45.999286 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:45.999676 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:45.999681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:45.999908 (+ 227us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.000059 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.000365 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260927 18:37:46.002431 8104 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36259
I20260927 18:37:46.002560 8104 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:46.003050 8104 heartbeater.cc:511] Master 127.6.4.62:36259 requested a full tablet report, sending...
I20260927 18:37:46.004489 7899 ts_manager.cc:194] Registered new tserver with Master: eb4fc38a6b2748699758376ce65801bc (127.6.4.1:39271)
I20260927 18:37:46.005824 7899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:34841
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:37:46.011197 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.004438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:49883 (local address 127.6.4.62:36259)
0927 18:37:46.004537 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.004540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.004752 (+ 212us) server_negotiation.cc:407] Connection header received
0927 18:37:46.004845 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.004848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.004885 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.004963 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.005405 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.006099 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.007358 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.007670 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.007758 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.008777 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.008780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.008782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.008802 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.009980 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.010325 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.010327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.010328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.010389 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.010556 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.010558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.010559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.010734 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.010807 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.010987 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.011075 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:37:46.011219 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.004368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.6.4.1:49883)
0927 18:37:46.004669 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.004683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.004772 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.004991 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.004994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.005186 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:46.005295 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.005301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.006232 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.006236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.007234 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.007246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.007771 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.007787 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.007875 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.008274 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:46.008288 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:46.008659 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:46.010069 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.010071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.010073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.010258 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.010438 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.010440 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.010441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.010498 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.010806 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:46.010808 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:46.010876 (+ 68us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.010949 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.011037 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:46.024497 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36259
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:44495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:46.125619 8113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:46.125836 8113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:46.125880 8113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:46.129684 8113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:46.129729 8113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:46.129765 8113 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:37:46.129782 8113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:37:46.129812 8113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:46.134657 8113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:44495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:36259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:46.135461 8113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:46.136325 8113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:46.144796 8119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:46.144863 8121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:46.144874 8118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:46.145216 8113 server_base.cc:1034] running on GCE node
I20260927 18:37:46.145550 8113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:46.146102 8113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:46.147249 8113 hybrid_clock.cc:648] HybridClock initialized: now 1790534266147228 us; error 32 us; skew 500 ppm
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534266, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:46.149631 8113 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:46.150411 8113 webserver.cc:460] Webserver started at http://127.6.4.2:38675/ using document root <none> and password file <none>
I20260927 18:37:46.150897 8113 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:46.150957 8113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:46.151122 8113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:46.153023 8113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "fd90722a0b49465cb893df2a27dd8280"
format_stamp: "Formatted at 2026-09-27 18:37:46 on dist-test-slave-bzzj"
server_key: "7fb5935d1cdd085a39ab923422bd26ce"
server_key_iv: "fde3500f7c6d2104b832571d59394d29"
server_key_version: "encryptionkey@0"
I20260927 18:37:46.153527 8113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "fd90722a0b49465cb893df2a27dd8280"
format_stamp: "Formatted at 2026-09-27 18:37:46 on dist-test-slave-bzzj"
server_key: "7fb5935d1cdd085a39ab923422bd26ce"
server_key_iv: "fde3500f7c6d2104b832571d59394d29"
server_key_version: "encryptionkey@0"
I20260927 18:37:46.156986 8113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:37:46.159152 8128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:46.160367 8113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:37:46.160461 8113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "fd90722a0b49465cb893df2a27dd8280"
format_stamp: "Formatted at 2026-09-27 18:37:46 on dist-test-slave-bzzj"
server_key: "7fb5935d1cdd085a39ab923422bd26ce"
server_key_iv: "fde3500f7c6d2104b832571d59394d29"
server_key_version: "encryptionkey@0"
I20260927 18:37:46.160542 8113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:46.184396 8113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:46.186961 8113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:46.187115 8113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:46.188453 8113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:46.188498 8113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:46.188548 8113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:46.188573 8113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534266, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:46.197829 8113 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:33275
I20260927 18:37:46.197902 8243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:33275 every 8 connection(s)
I20260927 18:37:46.198796 8113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:46.199223 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 8113
I20260927 18:37:46.199337 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:46.199622 6160 external_mini_cluster.cc:1428] Setting key 559fb97736f722701381b81e08970ce4
I20260927 18:37:46.201081 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.189062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:48333 (local address 127.6.4.62:36259)
0927 18:37:46.189199 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.189204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.190428 (+ 1224us) server_negotiation.cc:407] Connection header received
0927 18:37:46.191746 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.191750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.191807 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.191902 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.193026 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.193828 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.194991 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.195309 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.195420 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.197562 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.197565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.197568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.197595 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.199113 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.199661 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.199665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.199667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.199764 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.200137 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.200141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.200142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.200311 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.200395 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.200829 (+ 434us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.200936 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:37:46.201776 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.189853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.6.4.2:48333)
0927 18:37:46.190285 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.190330 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.191544 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.192074 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.192081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.192415 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:46.192905 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.192920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.193947 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.193951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.194894 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.194903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.195406 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.195444 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.195573 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.196073 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:46.196117 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:46.197415 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:46.199239 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.199244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.199257 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.199556 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.199854 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.199857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.199859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.200028 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.200411 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:46.200417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:46.200619 (+ 202us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.200792 (+ 173us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.201040 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260927 18:37:46.203207 8244 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36259
I20260927 18:37:46.203330 8244 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:46.203811 8244 heartbeater.cc:511] Master 127.6.4.62:36259 requested a full tablet report, sending...
I20260927 18:37:46.204833 7901 ts_manager.cc:194] Registered new tserver with Master: fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275)
I20260927 18:37:46.205479 7901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:48333
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:37:46.211887 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.205284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:39791 (local address 127.6.4.62:36259)
0927 18:37:46.205375 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.205379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.205506 (+ 127us) server_negotiation.cc:407] Connection header received
0927 18:37:46.205649 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.205652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.205698 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.205773 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.206306 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.207011 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.208158 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.208372 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.208448 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.209413 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.209416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.209417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.209437 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.210513 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.210841 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.210843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.210844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.210908 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.211156 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.211159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.211161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.211348 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.211430 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.211650 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.211761 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:37:46.211886 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.205206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.6.4.2:39791)
0927 18:37:46.205437 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.205450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.205538 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.205873 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.205877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.206063 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:46.206188 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.206195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.207146 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.207150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.208012 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.208022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.208463 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.208480 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.208573 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.208985 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:46.208999 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:46.209306 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:46.210599 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.210602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.210604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.210765 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.211002 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.211005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.211007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.211076 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.211434 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:46.211437 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:46.211514 (+ 77us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.211593 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.211710 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":169,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:46.224071 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36259
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:44495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:46.325006 8253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:46.325223 8253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:46.325280 8253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:46.329138 8253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:46.329185 8253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:46.329237 8253 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:37:46.329263 8253 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:37:46.329308 8253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:46.334211 8253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:44495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:36259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:46.335064 8253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:46.335923 8253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:46.344233 8261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:46.344228 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:46.344303 8253 server_base.cc:1034] running on GCE node
W20260927 18:37:46.344242 8259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:46.344748 8253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:46.345252 8253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:46.346396 8253 hybrid_clock.cc:648] HybridClock initialized: now 1790534266346382 us; error 25 us; skew 500 ppm
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534266, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:46.348809 8253 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:46.349478 8253 webserver.cc:460] Webserver started at http://127.6.4.3:37333/ using document root <none> and password file <none>
I20260927 18:37:46.349927 8253 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:46.349977 8253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:46.350126 8253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:46.351879 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "f23bf25d293a4aa6982e00efa0559ef2"
format_stamp: "Formatted at 2026-09-27 18:37:46 on dist-test-slave-bzzj"
server_key: "13bf200de9ff3a4ce969858a7541ebd6"
server_key_iv: "7bd9574df4958bb5ab0d972d4f8d044d"
server_key_version: "encryptionkey@0"
I20260927 18:37:46.352372 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "f23bf25d293a4aa6982e00efa0559ef2"
format_stamp: "Formatted at 2026-09-27 18:37:46 on dist-test-slave-bzzj"
server_key: "13bf200de9ff3a4ce969858a7541ebd6"
server_key_iv: "7bd9574df4958bb5ab0d972d4f8d044d"
server_key_version: "encryptionkey@0"
I20260927 18:37:46.355659 8253 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:46.357741 8268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:46.358812 8253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:46.358906 8253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "f23bf25d293a4aa6982e00efa0559ef2"
format_stamp: "Formatted at 2026-09-27 18:37:46 on dist-test-slave-bzzj"
server_key: "13bf200de9ff3a4ce969858a7541ebd6"
server_key_iv: "7bd9574df4958bb5ab0d972d4f8d044d"
server_key_version: "encryptionkey@0"
I20260927 18:37:46.358987 8253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:46.369038 8253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:46.371745 8253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:46.371891 8253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:46.373396 8253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:46.373449 8253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:46.373498 8253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:46.373539 8253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534266, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:46.383069 8253 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:44039
I20260927 18:37:46.383957 8253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:37:46.384236 8383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:44039 every 8 connection(s)
I20260927 18:37:46.386708 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.374888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:46631 (local address 127.6.4.62:36259)
0927 18:37:46.375281 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.375285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.377256 (+ 1971us) server_negotiation.cc:407] Connection header received
0927 18:37:46.377307 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.377310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.377365 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.377449 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.378890 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.379450 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.380761 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.380980 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.381078 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.383606 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.383609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.383611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.383634 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.385160 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.385584 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.385587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.385588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.385664 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.385909 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.385911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.385912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.386070 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.386140 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.386495 (+ 355us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.386592 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260927 18:37:46.387426 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.374960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.6.4.3:46631)
0927 18:37:46.375831 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.375883 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.376890 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.377641 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.377650 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.378083 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:46.378714 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.378733 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.379576 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.379580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.380641 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.380650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.381066 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.381118 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.381307 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.381792 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:46.381807 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:46.383420 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:46.385260 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.385264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.385274 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.385491 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.385731 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.385734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.385735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.385848 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.386137 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:46.386141 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:46.386329 (+ 188us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.386463 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.386674 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":125,"threads_started":1}
I20260927 18:37:46.388394 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 8253
I20260927 18:37:46.388496 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:46.388628 8384 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36259
I20260927 18:37:46.388756 8384 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:46.388778 6160 external_mini_cluster.cc:1428] Setting key 39950a27c3d51066c343afa05f6bc1fc
I20260927 18:37:46.389272 8384 heartbeater.cc:511] Master 127.6.4.62:36259 requested a full tablet report, sending...
I20260927 18:37:46.390420 7899 ts_manager.cc:194] Registered new tserver with Master: f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:46.391068 7899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:46631
I20260927 18:37:46.391283 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:37:46.397797 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.390674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.6.4.3:50291)
0927 18:37:46.390950 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.390964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.391081 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.391615 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.391618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.391820 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:46.391944 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.391952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.392945 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.392948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.393538 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.393546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.393929 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.393945 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.394029 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.394414 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:46.394427 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:46.394728 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:46.396420 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.396423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.396425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.396678 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.396957 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.396959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.396961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.397037 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.397407 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:46.397409 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:46.397484 (+ 75us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.397569 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.397662 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260927 18:37:46.397882 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.390710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:50291 (local address 127.6.4.62:36259)
0927 18:37:46.390805 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.390809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.391022 (+ 213us) server_negotiation.cc:407] Connection header received
0927 18:37:46.391162 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.391164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.391203 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.391275 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.392069 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.392807 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.393635 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.393853 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.393933 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.394842 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.394846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.394848 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.394873 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.396324 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.396780 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.396784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.396786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.396873 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.397116 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.397119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.397120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.397301 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.397389 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.397637 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.397741 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:46.414974 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.407801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50204 (local address 127.6.4.62:36259)
0927 18:37:46.407911 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.407914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.407924 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:37:46.407950 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.407952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.407982 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.408041 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.408687 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.409147 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.409932 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.410136 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.410217 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.412466 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.412469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.412472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.412493 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.413605 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.413963 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.413967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.413968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.414033 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.414264 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.414268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.414269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.414419 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.414548 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.414742 (+ 194us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.414839 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:37:46.417452 7899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:37:46.420272 7899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:46.433445 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.427726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.6.4.3:44039)
0927 18:37:46.428499 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.428510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.428531 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:37:46.428598 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.428603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.428781 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.429244 (+ 463us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:46.429362 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.430448 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.432046 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.432803 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.433024 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.433187 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.433235 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.433319 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":104,"threads_started":1}
I20260927 18:37:46.433439 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.428241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39271 (local address 127.0.0.1:57494)
0927 18:37:46.429098 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.429110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.429179 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.429562 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.429565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.429580 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:46.429676 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.429683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.430845 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.430848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.431877 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.431886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.432939 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.432943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.433064 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.433106 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.433185 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.433278 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":23,"thread_start_us":77,"threads_started":1}
I20260927 18:37:46.433449 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.427580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:44039 (local address 127.0.0.1:47774)
0927 18:37:46.427996 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.428055 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.428234 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.428943 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.428949 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.428974 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:46.429100 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.429108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.430555 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.430559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.431948 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.431958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.432939 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.432943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.433084 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.433112 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.433181 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.433263 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":86,"thread_start_us":49,"threads_started":1}
I20260927 18:37:46.433895 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.427924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33275 (local address 127.0.0.1:44228)
0927 18:37:46.428795 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.428808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.428890 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.429861 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.429863 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.429876 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:46.429931 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.429935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.430977 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.430980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.432157 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.432165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.432882 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.432885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.433319 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.433336 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.433394 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.433447 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":50,"thread_start_us":101,"threads_started":1}
I20260927 18:37:46.433957 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.428336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.6.4.2:33275)
0927 18:37:46.429365 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.429373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.429390 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:37:46.429443 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.429446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.429609 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.429747 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:46.430027 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.430844 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.432256 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.432779 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.432872 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.433021 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.433529 (+ 508us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.433836 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":953,"thread_start_us":91,"threads_started":1}
I20260927 18:37:46.434536 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.428619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.6.4.1:39271)
0927 18:37:46.429054 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.429065 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.429181 (+ 116us) server_negotiation.cc:407] Connection header received
0927 18:37:46.429257 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.429262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.429431 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.429583 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:46.429759 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.430719 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.431979 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.432803 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.433769 (+ 966us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.433915 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.434317 (+ 402us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.434410 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":244,"threads_started":1}
I20260927 18:37:46.436501 8038 tablet_service.cc:1467] Processing CreateTablet for tablet 32b1adcadc8c4d49baee28c6b6c07f75 (DEFAULT_TABLE table=test-table [id=5fbc3f6f14774b55af94dea3847adf1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:46.436412 8178 tablet_service.cc:1467] Processing CreateTablet for tablet 32b1adcadc8c4d49baee28c6b6c07f75 (DEFAULT_TABLE table=test-table [id=5fbc3f6f14774b55af94dea3847adf1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:46.436412 8307 tablet_service.cc:1467] Processing CreateTablet for tablet 32b1adcadc8c4d49baee28c6b6c07f75 (DEFAULT_TABLE table=test-table [id=5fbc3f6f14774b55af94dea3847adf1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:46.437322 8038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b1adcadc8c4d49baee28c6b6c07f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.437357 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b1adcadc8c4d49baee28c6b6c07f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.437363 8307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32b1adcadc8c4d49baee28c6b6c07f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.443255 8407 tablet_bootstrap.cc:492] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Bootstrap starting.
I20260927 18:37:46.444839 8408 tablet_bootstrap.cc:492] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Bootstrap starting.
I20260927 18:37:46.444839 8409 tablet_bootstrap.cc:492] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Bootstrap starting.
I20260927 18:37:46.445407 8407 tablet_bootstrap.cc:654] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.446051 8407 log.cc:824] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:46.446867 8409 tablet_bootstrap.cc:654] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.446867 8408 tablet_bootstrap.cc:654] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.447574 8407 tablet_bootstrap.cc:492] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: No bootstrap required, opened a new log
I20260927 18:37:46.447652 8408 log.cc:824] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:46.447654 8409 log.cc:824] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:46.447749 8407 ts_tablet_manager.cc:1397] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:37:46.449237 8408 tablet_bootstrap.cc:492] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: No bootstrap required, opened a new log
I20260927 18:37:46.449401 8408 ts_tablet_manager.cc:1397] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:37:46.449432 8409 tablet_bootstrap.cc:492] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: No bootstrap required, opened a new log
I20260927 18:37:46.449596 8409 ts_tablet_manager.cc:1397] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:37:46.451951 8407 raft_consensus.cc:348] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.452220 8407 raft_consensus.cc:374] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.452050 8408 raft_consensus.cc:348] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.452281 8407 raft_consensus.cc:729] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.452286 8408 raft_consensus.cc:374] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.452342 8408 raft_consensus.cc:729] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23bf25d293a4aa6982e00efa0559ef2, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.452718 8408 consensus_queue.cc:260] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.452711 8407 consensus_queue.cc:260] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.453050 8409 raft_consensus.cc:348] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.453272 8409 raft_consensus.cc:374] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.453325 8409 raft_consensus.cc:729] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90722a0b49465cb893df2a27dd8280, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.453593 8104 heartbeater.cc:503] Master 127.6.4.62:36259 was elected leader, sending a full tablet report...
I20260927 18:37:46.453621 8408 ts_tablet_manager.cc:1428] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260927 18:37:46.453629 8407 ts_tablet_manager.cc:1428] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260927 18:37:46.453679 8409 consensus_queue.cc:260] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.453826 8384 heartbeater.cc:503] Master 127.6.4.62:36259 was elected leader, sending a full tablet report...
I20260927 18:37:46.454290 8244 heartbeater.cc:503] Master 127.6.4.62:36259 was elected leader, sending a full tablet report...
I20260927 18:37:46.454445 8409 ts_tablet_manager.cc:1428] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260927 18:37:46.500614 8105 tablet.cc:2369] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:37:46.635388 8385 tablet.cc:2369] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:37:46.657310 8414 raft_consensus.cc:482] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:46.657498 8414 raft_consensus.cc:504] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.658564 8414 leader_election.cc:288] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039), fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275)
I20260927 18:37:46.662184 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.658903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:34601 (local address 127.6.4.3:44039)
0927 18:37:46.659015 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.659019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.659081 (+ 62us) server_negotiation.cc:407] Connection header received
0927 18:37:46.659222 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.659225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.659268 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.659327 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:46.659623 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.660188 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.661285 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.661681 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.661759 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.661820 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.661976 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.662048 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:37:46.662197 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.658795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33275 (local address 127.6.4.1:39893)
0927 18:37:46.659084 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.659097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.659191 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.659410 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.659413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.659428 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:46.659528 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.659533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.660306 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.660309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.661176 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.661185 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.661817 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.661820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.661908 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.661923 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.661992 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.662051 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":58,"thread_start_us":85,"threads_started":1}
I20260927 18:37:46.662205 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.658985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:39893 (local address 127.6.4.2:33275)
0927 18:37:46.659088 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.659092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.659163 (+ 71us) server_negotiation.cc:407] Connection header received
0927 18:37:46.659282 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.659285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.659329 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.659389 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:46.659664 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.660204 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.661283 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.661712 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.661808 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.661864 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.662014 (+ 150us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.662081 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:37:46.662216 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.658733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:44039 (local address 127.6.4.1:34601)
0927 18:37:46.659014 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.659027 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.659127 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.659345 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.659348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.659362 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:46.659510 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.659518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.660282 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.660285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.661173 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.661183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.661778 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.661782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.661869 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.661892 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.661948 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.662060 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":49,"thread_start_us":79,"threads_started":1}
I20260927 18:37:46.662783 8198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1adcadc8c4d49baee28c6b6c07f75" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280" is_pre_election: true
I20260927 18:37:46.662969 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1adcadc8c4d49baee28c6b6c07f75" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2" is_pre_election: true
I20260927 18:37:46.663053 8198 raft_consensus.cc:2454] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:46.663232 8338 raft_consensus.cc:2454] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:46.663532 7989 leader_election.cc:302] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, fd90722a0b49465cb893df2a27dd8280; no voters:
I20260927 18:37:46.663803 8414 raft_consensus.cc:2781] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:46.663875 8414 raft_consensus.cc:482] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:46.663906 8414 raft_consensus.cc:3037] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.664983 8414 raft_consensus.cc:504] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.665301 8414 leader_election.cc:288] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Requested vote from peers f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039), fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275)
I20260927 18:37:46.665585 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1adcadc8c4d49baee28c6b6c07f75" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2"
I20260927 18:37:46.665602 8198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32b1adcadc8c4d49baee28c6b6c07f75" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280"
I20260927 18:37:46.665679 8338 raft_consensus.cc:3037] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.665723 8198 raft_consensus.cc:3037] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.666738 8198 raft_consensus.cc:2454] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:46.666865 8338 raft_consensus.cc:2454] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:46.667013 7989 leader_election.cc:302] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, fd90722a0b49465cb893df2a27dd8280; no voters:
I20260927 18:37:46.667167 8414 raft_consensus.cc:2781] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:46.667321 8414 raft_consensus.cc:686] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Becoming Leader. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Running, Role: LEADER
I20260927 18:37:46.667596 8414 consensus_queue.cc:237] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } }
I20260927 18:37:46.670388 7901 catalog_manager.cc:5495] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc reported cstate change: term changed from 0 to 1, leader changed from <none> to eb4fc38a6b2748699758376ce65801bc (127.6.4.1). New cstate: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } health_report { overall_health: UNKNOWN } } }
W20260927 18:37:46.700125 8245 tablet.cc:2369] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:37:46 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534265, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:46.741257 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.733451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36259 (local address 127.0.0.1:50220)
0927 18:37:46.733698 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.733709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.733796 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.733979 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.733981 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.734346 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:46.734435 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.734440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.735347 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.735349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.735895 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.735902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.736396 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.736409 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.736466 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.736996 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:46.737007 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:46.738356 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:46.739773 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.739781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.739786 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.740050 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.740295 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:46.740298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:46.740300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:46.740379 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:46.740691 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:46.740697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:46.740810 (+ 113us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.740909 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.741040 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260927 18:37:46.741268 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.733533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.6.4.62:36259)
0927 18:37:46.733652 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.733656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.733765 (+ 109us) server_negotiation.cc:407] Connection header received
0927 18:37:46.733860 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.733862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.733898 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.733957 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:46.734519 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.735250 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.735991 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.736319 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.736418 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.738472 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:46.738475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:46.738477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:46.738497 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:46.739683 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.740164 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.740166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.740168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.740229 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:46.740442 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:46.740445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:46.740446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:46.740583 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:46.740676 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.740954 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.741088 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":46}
I20260927 18:37:46.744777 7901 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:50220:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:37:46.745476 7901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:46.749640 8038 tablet_service.cc:1467] Processing CreateTablet for tablet 74a2463d72d34038a048c66582dc117a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13880f697b3046768d65297f3ac2d730]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:37:46.749630 8178 tablet_service.cc:1467] Processing CreateTablet for tablet 74a2463d72d34038a048c66582dc117a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13880f697b3046768d65297f3ac2d730]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:37:46.750312 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a2463d72d34038a048c66582dc117a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.750312 8038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a2463d72d34038a048c66582dc117a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.749928 8307 tablet_service.cc:1467] Processing CreateTablet for tablet 74a2463d72d34038a048c66582dc117a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13880f697b3046768d65297f3ac2d730]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:37:46.750519 8307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74a2463d72d34038a048c66582dc117a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.752988 8409 tablet_bootstrap.cc:492] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: Bootstrap starting.
I20260927 18:37:46.752982 8408 tablet_bootstrap.cc:492] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: Bootstrap starting.
I20260927 18:37:46.753224 8407 tablet_bootstrap.cc:492] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: Bootstrap starting.
I20260927 18:37:46.753901 8408 tablet_bootstrap.cc:654] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.753901 8409 tablet_bootstrap.cc:654] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.754281 8407 tablet_bootstrap.cc:654] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.755035 8409 tablet_bootstrap.cc:492] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: No bootstrap required, opened a new log
I20260927 18:37:46.755115 8409 ts_tablet_manager.cc:1397] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:46.755371 8407 tablet_bootstrap.cc:492] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: No bootstrap required, opened a new log
I20260927 18:37:46.755441 8407 ts_tablet_manager.cc:1397] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260927 18:37:46.755774 8409 raft_consensus.cc:348] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.755852 8409 raft_consensus.cc:374] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.755801 8407 raft_consensus.cc:348] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.755878 8409 raft_consensus.cc:729] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90722a0b49465cb893df2a27dd8280, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.755903 8407 raft_consensus.cc:374] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.755924 8407 raft_consensus.cc:729] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.755982 8409 consensus_queue.cc:260] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.756032 8407 consensus_queue.cc:260] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.756246 8409 ts_tablet_manager.cc:1428] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:46.756222 8415 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.756301 8407 ts_tablet_manager.cc:1428] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:46.756271 8414 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.756340 8415 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.756379 8414 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.757061 8408 tablet_bootstrap.cc:492] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: No bootstrap required, opened a new log
I20260927 18:37:46.757148 8408 ts_tablet_manager.cc:1397] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 18:37:46.757566 8408 raft_consensus.cc:348] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.757676 8408 raft_consensus.cc:374] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.757727 8408 raft_consensus.cc:729] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23bf25d293a4aa6982e00efa0559ef2, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.757848 8408 consensus_queue.cc:260] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.758088 8408 ts_tablet_manager.cc:1428] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:37:46.758076 8413 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.758204 8413 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.784425 8414 raft_consensus.cc:482] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:46.784526 8414 raft_consensus.cc:504] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.784842 8414 leader_election.cc:288] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:46.785179 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a2463d72d34038a048c66582dc117a" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2" is_pre_election: true
I20260927 18:37:46.785202 8198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a2463d72d34038a048c66582dc117a" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280" is_pre_election: true
I20260927 18:37:46.785296 8338 raft_consensus.cc:2454] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:46.785327 8198 raft_consensus.cc:2454] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:46.785632 7992 leader_election.cc:302] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, f23bf25d293a4aa6982e00efa0559ef2; no voters:
I20260927 18:37:46.785789 8414 raft_consensus.cc:2781] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:46.785856 8414 raft_consensus.cc:482] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:46.785899 8414 raft_consensus.cc:3037] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.786758 8414 raft_consensus.cc:504] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.787119 8414 leader_election.cc:288] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Requested vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:46.787431 8198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a2463d72d34038a048c66582dc117a" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280"
I20260927 18:37:46.787463 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74a2463d72d34038a048c66582dc117a" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2"
I20260927 18:37:46.787540 8198 raft_consensus.cc:3037] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.787550 8338 raft_consensus.cc:3037] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.788618 8338 raft_consensus.cc:2454] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:46.788618 8198 raft_consensus.cc:2454] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:46.788847 7989 leader_election.cc:302] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, fd90722a0b49465cb893df2a27dd8280; no voters:
I20260927 18:37:46.788992 8414 raft_consensus.cc:2781] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:46.789076 8414 raft_consensus.cc:686] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Becoming Leader. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Running, Role: LEADER
I20260927 18:37:46.789233 8414 consensus_queue.cc:237] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.789829 8418 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: TxnStatusTablet state changed. Reason: New leader eb4fc38a6b2748699758376ce65801bc. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.789924 8418 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:37:46.790230 8430 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:37:46.790437 8430 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:37:46.790731 7901 catalog_manager.cc:5495] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc reported cstate change: term changed from 0 to 1, leader changed from <none> to eb4fc38a6b2748699758376ce65801bc (127.6.4.1). New cstate: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } health_report { overall_health: UNKNOWN } } }
I20260927 18:37:46.806083 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.802677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57498 (local address 127.6.4.1:39271)
0927 18:37:46.802797 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.802802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.802833 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:37:46.803048 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.803051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.803107 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.803182 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:46.803408 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.803895 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.804788 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.805000 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.805085 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.805347 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:46.805753 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:46.805850 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.805912 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.805961 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:37:46.806093 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.802634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39271 (local address 127.0.0.1:57498)
0927 18:37:46.802775 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.802790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.802906 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.803212 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.803214 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.803222 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:37:46.803316 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.803322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.804021 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.804024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.804681 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.804691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.805081 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.805095 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.805187 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.805228 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:37:46.805827 (+ 599us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:37:46.805842 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.805905 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.805985 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:37:46.810418 8198 raft_consensus.cc:1261] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:46.810418 8338 raft_consensus.cc:1261] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:46.811043 8418 consensus_queue.cc:1035] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:46.811189 8414 consensus_queue.cc:1035] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:46.815124 8413 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: TxnStatusTablet state changed. Reason: New leader eb4fc38a6b2748699758376ce65801bc. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.815253 8413 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.815284 8415 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: TxnStatusTablet state changed. Reason: New leader eb4fc38a6b2748699758376ce65801bc. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.815378 8415 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.817301 8418 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.817406 8418 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:37:46.817598 8433 mvcc.cc:205] Tried to move back new op lower bound from 7334028356849811456 to 7334028356769755136. Current Snapshot: MvccSnapshot[applied={T|T < 7334028356849811456}]
I20260927 18:37:46.818109 8415 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.818197 8415 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P fd90722a0b49465cb893df2a27dd8280: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.818367 8414 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.818449 8414 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P eb4fc38a6b2748699758376ce65801bc: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:37:46.819279 8413 tablet_replica.cc:439] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } } }
I20260927 18:37:46.819372 8413 tablet_replica.cc:442] T 74a2463d72d34038a048c66582dc117a P f23bf25d293a4aa6982e00efa0559ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:37:46.820948 8434 mvcc.cc:205] Tried to move back new op lower bound from 7334028356849811456 to 7334028356769755136. Current Snapshot: MvccSnapshot[applied={T|T < 7334028356849811456}]
I20260927 18:37:46.822607 8436 mvcc.cc:205] Tried to move back new op lower bound from 7334028356849811456 to 7334028356769755136. Current Snapshot: MvccSnapshot[applied={T|T < 7334028356849811456}]
I20260927 18:37:46.828469 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.825597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.6.4.1:39271)
0927 18:37:46.825732 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.825736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.825767 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:37:46.825898 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.825901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.825941 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.825997 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:46.826240 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.826720 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.827526 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.827741 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.827819 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.828066 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:46.828162 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:46.828218 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.828313 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.828357 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:37:46.838392 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.835523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:60219 (local address 127.6.4.1:39271)
0927 18:37:46.835613 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:46.835618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:46.835767 (+ 149us) server_negotiation.cc:407] Connection header received
0927 18:37:46.835874 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:46.835877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:46.835911 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:46.835963 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:46.836196 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.836642 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.837484 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.837671 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.837748 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.838008 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:46.838076 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:46.838128 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:46.838221 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 18:37:46.838259 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":30}
I20260927 18:37:46.838400 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:46.835475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39271 (local address 127.6.4.1:60219)
0927 18:37:46.835699 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:46.835710 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:46.835796 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:46.835975 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:46.835978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:46.835998 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:37:46.836083 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.836089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.836744 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.836747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.837383 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:46.837391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:46.837742 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:46.837755 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:46.837849 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:46.837884 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:37:46.838140 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:37:46.838146 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:37:46.838199 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:37:46.838265 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260927 18:37:46.844152 8338 raft_consensus.cc:1261] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:46.844228 8198 raft_consensus.cc:1261] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:46.844480 8439 consensus_queue.cc:1035] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:46.844609 8414 consensus_queue.cc:1035] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:46.880811 7901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:46.881009 7901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:46.883219 7901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Sending DeleteTablet for 3 replicas of tablet 32b1adcadc8c4d49baee28c6b6c07f75
I20260927 18:37:46.883882 8307 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b1adcadc8c4d49baee28c6b6c07f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:46 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:47774
I20260927 18:37:46.884016 8178 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b1adcadc8c4d49baee28c6b6c07f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:46 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:44228
I20260927 18:37:46.884222 8037 tablet_service.cc:1514] Processing DeleteTablet for tablet 32b1adcadc8c4d49baee28c6b6c07f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:46 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:57494
I20260927 18:37:46.884419 7901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:37:46.884733 8451 tablet_replica.cc:330] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: stopping tablet replica
W20260927 18:37:46.884820 7901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:46.885005 8451 raft_consensus.cc:2229] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:37:46.885308 8451 raft_consensus.cc:2258] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:46.885569 8452 tablet_replica.cc:330] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: stopping tablet replica
I20260927 18:37:46.885838 8452 raft_consensus.cc:2229] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:46.886039 8452 raft_consensus.cc:2258] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:46.886353 8450 tablet_replica.cc:330] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: stopping tablet replica
I20260927 18:37:46.886582 8450 raft_consensus.cc:2229] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:46.886751 8450 raft_consensus.cc:2258] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:46.888229 8452 ts_tablet_manager.cc:1905] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:46.889240 8450 ts_tablet_manager.cc:1905] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:46.889413 8037 tablet_service.cc:1467] Processing CreateTablet for tablet c2336acf53e64c4eb6f806396a70c056 (DEFAULT_TABLE table=test-table [id=110e8b8e735642debcef05bf5a4a0e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:46.889422 8178 tablet_service.cc:1467] Processing CreateTablet for tablet c2336acf53e64c4eb6f806396a70c056 (DEFAULT_TABLE table=test-table [id=110e8b8e735642debcef05bf5a4a0e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:46.889653 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2336acf53e64c4eb6f806396a70c056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.889658 8037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2336acf53e64c4eb6f806396a70c056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.890933 8307 tablet_service.cc:1467] Processing CreateTablet for tablet c2336acf53e64c4eb6f806396a70c056 (DEFAULT_TABLE table=test-table [id=110e8b8e735642debcef05bf5a4a0e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:46.891166 8307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2336acf53e64c4eb6f806396a70c056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:46.892169 8450 ts_tablet_manager.cc:1918] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:46.892230 8409 tablet_bootstrap.cc:492] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Bootstrap starting.
I20260927 18:37:46.892354 8450 log.cc:1196] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/32b1adcadc8c4d49baee28c6b6c07f75
I20260927 18:37:46.892720 8450 ts_tablet_manager.cc:1939] T 32b1adcadc8c4d49baee28c6b6c07f75 P fd90722a0b49465cb893df2a27dd8280: Deleting consensus metadata
I20260927 18:37:46.893113 8409 tablet_bootstrap.cc:654] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.893282 8451 ts_tablet_manager.cc:1905] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:46.893373 8408 tablet_bootstrap.cc:492] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Bootstrap starting.
I20260927 18:37:46.894033 7885 catalog_manager.cc:4841] TS fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275): tablet 32b1adcadc8c4d49baee28c6b6c07f75 (table test-table [id=5fbc3f6f14774b55af94dea3847adf1a]) successfully deleted
I20260927 18:37:46.894207 8408 tablet_bootstrap.cc:654] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.896047 8407 tablet_bootstrap.cc:492] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Bootstrap starting.
I20260927 18:37:46.896080 8451 ts_tablet_manager.cc:1918] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:46.896163 8451 log.cc:1196] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/32b1adcadc8c4d49baee28c6b6c07f75
I20260927 18:37:46.896421 8451 ts_tablet_manager.cc:1939] T 32b1adcadc8c4d49baee28c6b6c07f75 P eb4fc38a6b2748699758376ce65801bc: Deleting consensus metadata
I20260927 18:37:46.896868 8407 tablet_bootstrap.cc:654] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:46.897392 7888 catalog_manager.cc:4841] TS eb4fc38a6b2748699758376ce65801bc (127.6.4.1:39271): tablet 32b1adcadc8c4d49baee28c6b6c07f75 (table test-table [id=5fbc3f6f14774b55af94dea3847adf1a]) successfully deleted
I20260927 18:37:46.898167 8452 ts_tablet_manager.cc:1918] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:46.898257 8452 log.cc:1196] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/32b1adcadc8c4d49baee28c6b6c07f75
I20260927 18:37:46.898325 8407 tablet_bootstrap.cc:492] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: No bootstrap required, opened a new log
I20260927 18:37:46.898406 8407 ts_tablet_manager.cc:1397] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:46.898523 8452 ts_tablet_manager.cc:1939] T 32b1adcadc8c4d49baee28c6b6c07f75 P f23bf25d293a4aa6982e00efa0559ef2: Deleting consensus metadata
I20260927 18:37:46.898787 8407 raft_consensus.cc:348] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.898875 8407 raft_consensus.cc:374] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.898903 8407 raft_consensus.cc:729] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.899010 8407 consensus_queue.cc:260] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.899231 8407 ts_tablet_manager.cc:1428] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:46.899559 7888 catalog_manager.cc:4841] TS f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039): tablet 32b1adcadc8c4d49baee28c6b6c07f75 (table test-table [id=5fbc3f6f14774b55af94dea3847adf1a]) successfully deleted
I20260927 18:37:46.899981 8408 tablet_bootstrap.cc:492] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: No bootstrap required, opened a new log
I20260927 18:37:46.900061 8408 ts_tablet_manager.cc:1397] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260927 18:37:46.900475 8408 raft_consensus.cc:348] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.900580 8408 raft_consensus.cc:374] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.900615 8408 raft_consensus.cc:729] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23bf25d293a4aa6982e00efa0559ef2, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.900733 8408 consensus_queue.cc:260] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.900954 8408 ts_tablet_manager.cc:1428] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:37:46.904691 8409 tablet_bootstrap.cc:492] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: No bootstrap required, opened a new log
I20260927 18:37:46.904770 8409 ts_tablet_manager.cc:1397] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20260927 18:37:46.905169 8409 raft_consensus.cc:348] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.905280 8409 raft_consensus.cc:374] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:46.905313 8409 raft_consensus.cc:729] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90722a0b49465cb893df2a27dd8280, State: Initialized, Role: FOLLOWER
I20260927 18:37:46.905426 8409 consensus_queue.cc:260] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.905663 8409 ts_tablet_manager.cc:1428] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:37:46.987735 8414 raft_consensus.cc:482] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:46.987866 8414 raft_consensus.cc:504] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.988301 8414 leader_election.cc:288] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:46.988667 8198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2336acf53e64c4eb6f806396a70c056" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280" is_pre_election: true
I20260927 18:37:46.988673 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2336acf53e64c4eb6f806396a70c056" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2" is_pre_election: true
I20260927 18:37:46.988807 8198 raft_consensus.cc:2454] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:46.988812 8338 raft_consensus.cc:2454] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:46.989070 7989 leader_election.cc:302] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, f23bf25d293a4aa6982e00efa0559ef2, fd90722a0b49465cb893df2a27dd8280; no voters:
I20260927 18:37:46.989287 8414 raft_consensus.cc:2781] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:46.989339 8414 raft_consensus.cc:482] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:46.989377 8414 raft_consensus.cc:3037] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.990269 8414 raft_consensus.cc:504] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.990566 8414 leader_election.cc:288] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Requested vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:46.990839 8198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2336acf53e64c4eb6f806396a70c056" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280"
I20260927 18:37:46.990976 8198 raft_consensus.cc:3037] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.990963 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2336acf53e64c4eb6f806396a70c056" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2"
I20260927 18:37:46.991051 8338 raft_consensus.cc:3037] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:46.991983 8198 raft_consensus.cc:2454] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:46.991983 8338 raft_consensus.cc:2454] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:46.992267 7992 leader_election.cc:302] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, f23bf25d293a4aa6982e00efa0559ef2; no voters:
I20260927 18:37:46.992421 8414 raft_consensus.cc:2781] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:46.992512 8414 raft_consensus.cc:686] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Becoming Leader. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Running, Role: LEADER
I20260927 18:37:46.992641 8414 consensus_queue.cc:237] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:46.994139 7901 catalog_manager.cc:5495] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc reported cstate change: term changed from 0 to 1, leader changed from <none> to eb4fc38a6b2748699758376ce65801bc (127.6.4.1). New cstate: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } health_report { overall_health: UNKNOWN } } }
I20260927 18:37:47.024861 8338 raft_consensus.cc:1261] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:47.025054 8198 raft_consensus.cc:1261] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:47.025231 8414 consensus_queue.cc:1035] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:47.025403 8418 consensus_queue.cc:1035] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:37:47.038517 8433 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 18:37:48.583724 7901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:48.583863 7901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:48.585954 7901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Sending DeleteTablet for 3 replicas of tablet c2336acf53e64c4eb6f806396a70c056
I20260927 18:37:48.586369 8038 tablet_service.cc:1514] Processing DeleteTablet for tablet c2336acf53e64c4eb6f806396a70c056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:48 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:57494
I20260927 18:37:48.586439 8178 tablet_service.cc:1514] Processing DeleteTablet for tablet c2336acf53e64c4eb6f806396a70c056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:48 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:44228
I20260927 18:37:48.586643 8307 tablet_service.cc:1514] Processing DeleteTablet for tablet c2336acf53e64c4eb6f806396a70c056 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:48 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:47774
I20260927 18:37:48.586807 8480 tablet_replica.cc:330] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: stopping tablet replica
I20260927 18:37:48.586911 8480 raft_consensus.cc:2229] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:48.587000 8480 raft_consensus.cc:2258] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:48.587172 8481 tablet_replica.cc:330] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: stopping tablet replica
I20260927 18:37:48.587321 8481 raft_consensus.cc:2229] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:48.587436 8481 raft_consensus.cc:2258] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:48.587548 8480 ts_tablet_manager.cc:1905] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:48.587927 8481 ts_tablet_manager.cc:1905] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:48.586680 8479 tablet_replica.cc:330] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: stopping tablet replica
I20260927 18:37:48.589279 8479 raft_consensus.cc:2229] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:37:48.589454 8479 raft_consensus.cc:2258] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:48.589994 8479 ts_tablet_manager.cc:1905] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:48.590201 8480 ts_tablet_manager.cc:1918] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:37:48.590266 8480 log.cc:1196] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/c2336acf53e64c4eb6f806396a70c056
I20260927 18:37:48.590443 8481 ts_tablet_manager.cc:1918] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:37:48.590512 8481 log.cc:1196] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/c2336acf53e64c4eb6f806396a70c056
I20260927 18:37:48.590507 8480 ts_tablet_manager.cc:1939] T c2336acf53e64c4eb6f806396a70c056 P fd90722a0b49465cb893df2a27dd8280: Deleting consensus metadata
I20260927 18:37:48.590797 8481 ts_tablet_manager.cc:1939] T c2336acf53e64c4eb6f806396a70c056 P f23bf25d293a4aa6982e00efa0559ef2: Deleting consensus metadata
I20260927 18:37:48.591066 7885 catalog_manager.cc:4841] TS fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275): tablet c2336acf53e64c4eb6f806396a70c056 (table test-table [id=110e8b8e735642debcef05bf5a4a0e85]) successfully deleted
I20260927 18:37:48.591444 7888 catalog_manager.cc:4841] TS f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039): tablet c2336acf53e64c4eb6f806396a70c056 (table test-table [id=110e8b8e735642debcef05bf5a4a0e85]) successfully deleted
I20260927 18:37:48.592254 8479 ts_tablet_manager.cc:1918] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:37:48.592312 8479 log.cc:1196] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/c2336acf53e64c4eb6f806396a70c056
I20260927 18:37:48.592512 8479 ts_tablet_manager.cc:1939] T c2336acf53e64c4eb6f806396a70c056 P eb4fc38a6b2748699758376ce65801bc: Deleting consensus metadata
I20260927 18:37:48.593134 7888 catalog_manager.cc:4841] TS eb4fc38a6b2748699758376ce65801bc (127.6.4.1:39271): tablet c2336acf53e64c4eb6f806396a70c056 (table test-table [id=110e8b8e735642debcef05bf5a4a0e85]) successfully deleted
Sep 27 18:37:48 dist-test-slave-bzzj krb5kdc[7856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534268, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:37:48 dist-test-slave-bzzj krb5kdc[7856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534268, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:48.613842 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:48.606263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.6.4.62:36259)
0927 18:37:48.606468 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:48.606474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:48.606503 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:37:48.606612 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:48.606614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:48.606654 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:48.606728 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:48.607331 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:48.607857 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:48.608679 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:48.608893 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:48.608965 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:48.611020 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:48.611023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:48.611025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:48.611047 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:48.612396 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:48.612758 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:48.612762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:48.612763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:48.612838 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:48.613068 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:48.613071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:48.613073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:48.613242 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:48.613354 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:48.613577 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 18:37:48.613687 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260927 18:37:48.616047 7901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:37:48.616459 7901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:48.620430 8038 tablet_service.cc:1467] Processing CreateTablet for tablet 27d13d8ca09a40d6810b54167949f77f (DEFAULT_TABLE table=test-table [id=39d7792437774d2f81b97a3f2f7f3f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:48.620451 8178 tablet_service.cc:1467] Processing CreateTablet for tablet 27d13d8ca09a40d6810b54167949f77f (DEFAULT_TABLE table=test-table [id=39d7792437774d2f81b97a3f2f7f3f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:48.620689 8038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d13d8ca09a40d6810b54167949f77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:48.620712 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d13d8ca09a40d6810b54167949f77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:48.620699 8307 tablet_service.cc:1467] Processing CreateTablet for tablet 27d13d8ca09a40d6810b54167949f77f (DEFAULT_TABLE table=test-table [id=39d7792437774d2f81b97a3f2f7f3f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:48.620961 8307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d13d8ca09a40d6810b54167949f77f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:48.622977 8490 tablet_bootstrap.cc:492] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Bootstrap starting.
I20260927 18:37:48.622989 8492 tablet_bootstrap.cc:492] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Bootstrap starting.
I20260927 18:37:48.623811 8492 tablet_bootstrap.cc:654] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:48.623811 8490 tablet_bootstrap.cc:654] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:48.624264 8491 tablet_bootstrap.cc:492] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Bootstrap starting.
I20260927 18:37:48.624991 8491 tablet_bootstrap.cc:654] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:48.625267 8492 tablet_bootstrap.cc:492] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: No bootstrap required, opened a new log
I20260927 18:37:48.625351 8492 ts_tablet_manager.cc:1397] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:48.625779 8492 raft_consensus.cc:348] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.625875 8492 raft_consensus.cc:374] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:48.625905 8492 raft_consensus.cc:729] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90722a0b49465cb893df2a27dd8280, State: Initialized, Role: FOLLOWER
I20260927 18:37:48.626019 8492 consensus_queue.cc:260] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.626214 8490 tablet_bootstrap.cc:492] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: No bootstrap required, opened a new log
I20260927 18:37:48.626281 8490 ts_tablet_manager.cc:1397] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:37:48.626384 8492 ts_tablet_manager.cc:1428] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:48.626674 8490 raft_consensus.cc:348] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.626791 8490 raft_consensus.cc:374] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:48.626850 8490 raft_consensus.cc:729] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23bf25d293a4aa6982e00efa0559ef2, State: Initialized, Role: FOLLOWER
I20260927 18:37:48.626992 8490 consensus_queue.cc:260] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.627202 8491 tablet_bootstrap.cc:492] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: No bootstrap required, opened a new log
I20260927 18:37:48.627326 8491 ts_tablet_manager.cc:1397] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:37:48.627350 8490 ts_tablet_manager.cc:1428] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:48.627715 8491 raft_consensus.cc:348] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.627871 8491 raft_consensus.cc:374] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:48.627918 8491 raft_consensus.cc:729] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Initialized, Role: FOLLOWER
I20260927 18:37:48.628074 8491 consensus_queue.cc:260] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.628449 8491 ts_tablet_manager.cc:1428] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:48.756071 8418 raft_consensus.cc:482] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:48.756229 8418 raft_consensus.cc:504] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.756574 8418 leader_election.cc:288] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:48.756934 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d13d8ca09a40d6810b54167949f77f" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2" is_pre_election: true
I20260927 18:37:48.756949 8197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d13d8ca09a40d6810b54167949f77f" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280" is_pre_election: true
I20260927 18:37:48.757054 8338 raft_consensus.cc:2454] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:48.757068 8197 raft_consensus.cc:2454] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:48.757336 7992 leader_election.cc:302] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, f23bf25d293a4aa6982e00efa0559ef2; no voters:
I20260927 18:37:48.757519 8418 raft_consensus.cc:2781] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:48.757572 8418 raft_consensus.cc:482] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:48.757614 8418 raft_consensus.cc:3037] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:48.758529 8418 raft_consensus.cc:504] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.758811 8418 leader_election.cc:288] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Requested vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:48.759109 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d13d8ca09a40d6810b54167949f77f" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2"
I20260927 18:37:48.759119 8197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d13d8ca09a40d6810b54167949f77f" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280"
I20260927 18:37:48.759209 8197 raft_consensus.cc:3037] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:48.759224 8338 raft_consensus.cc:3037] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:48.760051 8197 raft_consensus.cc:2454] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:48.760133 8338 raft_consensus.cc:2454] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:48.760349 7989 leader_election.cc:302] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, fd90722a0b49465cb893df2a27dd8280; no voters:
I20260927 18:37:48.760522 8418 raft_consensus.cc:2781] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:48.760604 8418 raft_consensus.cc:686] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Becoming Leader. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Running, Role: LEADER
I20260927 18:37:48.760730 8418 consensus_queue.cc:237] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.762145 7900 catalog_manager.cc:5495] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc reported cstate change: term changed from 0 to 1, leader changed from <none> to eb4fc38a6b2748699758376ce65801bc (127.6.4.1). New cstate: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } health_report { overall_health: UNKNOWN } } }
I20260927 18:37:48.778894 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:48.776308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57516 (local address 127.6.4.1:39271)
0927 18:37:48.776535 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:48.776541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:48.776552 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:37:48.776586 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:48.776588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:48.776624 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:48.776696 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:37:48.776872 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:48.777361 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:48.778115 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:48.778323 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:48.778389 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:48.778562 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:37:48.778620 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:37:48.778665 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:48.778724 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 18:37:48.778763 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
I20260927 18:37:48.783623 8338 raft_consensus.cc:1261] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:48.783695 8197 raft_consensus.cc:1261] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:48.783933 8473 consensus_queue.cc:1035] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:48.784061 8418 consensus_queue.cc:1035] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:48.809998 7900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:48.810117 7900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:48.811694 7900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Sending DeleteTablet for 3 replicas of tablet 27d13d8ca09a40d6810b54167949f77f
I20260927 18:37:48.812112 8037 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d13d8ca09a40d6810b54167949f77f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:48 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:57494
I20260927 18:37:48.812136 8178 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d13d8ca09a40d6810b54167949f77f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:48 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:44228
I20260927 18:37:48.812281 8480 tablet_replica.cc:330] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: stopping tablet replica
I20260927 18:37:48.812280 8307 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d13d8ca09a40d6810b54167949f77f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:48 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:47774
I20260927 18:37:48.812386 8480 raft_consensus.cc:2229] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:48.812422 8479 tablet_replica.cc:330] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: stopping tablet replica
I20260927 18:37:48.812505 8480 raft_consensus.cc:2258] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:48.812511 8479 raft_consensus.cc:2229] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:37:48.812645 8479 raft_consensus.cc:2258] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:48.812804 8481 tablet_replica.cc:330] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: stopping tablet replica
I20260927 18:37:48.812819 7900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:37:48.812909 8481 raft_consensus.cc:2229] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:48.813006 8481 raft_consensus.cc:2258] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:48.813090 8480 ts_tablet_manager.cc:1905] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:37:48.813171 7900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:48.813613 8479 ts_tablet_manager.cc:1905] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:48.814486 8481 ts_tablet_manager.cc:1905] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:48.815421 8480 ts_tablet_manager.cc:1918] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:48.815501 8480 log.cc:1196] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/27d13d8ca09a40d6810b54167949f77f
I20260927 18:37:48.815727 8480 ts_tablet_manager.cc:1939] T 27d13d8ca09a40d6810b54167949f77f P fd90722a0b49465cb893df2a27dd8280: Deleting consensus metadata
I20260927 18:37:48.816001 8479 ts_tablet_manager.cc:1918] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:48.816058 8479 log.cc:1196] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/27d13d8ca09a40d6810b54167949f77f
I20260927 18:37:48.816299 8479 ts_tablet_manager.cc:1939] T 27d13d8ca09a40d6810b54167949f77f P eb4fc38a6b2748699758376ce65801bc: Deleting consensus metadata
I20260927 18:37:48.816504 8481 ts_tablet_manager.cc:1918] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:37:48.816550 8481 log.cc:1196] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/27d13d8ca09a40d6810b54167949f77f
I20260927 18:37:48.816695 8481 ts_tablet_manager.cc:1939] T 27d13d8ca09a40d6810b54167949f77f P f23bf25d293a4aa6982e00efa0559ef2: Deleting consensus metadata
I20260927 18:37:48.816897 7885 catalog_manager.cc:4841] TS fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275): tablet 27d13d8ca09a40d6810b54167949f77f (table test-table [id=39d7792437774d2f81b97a3f2f7f3f5e]) successfully deleted
I20260927 18:37:48.816900 7888 catalog_manager.cc:4841] TS eb4fc38a6b2748699758376ce65801bc (127.6.4.1:39271): tablet 27d13d8ca09a40d6810b54167949f77f (table test-table [id=39d7792437774d2f81b97a3f2f7f3f5e]) successfully deleted
I20260927 18:37:48.817759 7888 catalog_manager.cc:4841] TS f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039): tablet 27d13d8ca09a40d6810b54167949f77f (table test-table [id=39d7792437774d2f81b97a3f2f7f3f5e]) successfully deleted
I20260927 18:37:48.818408 8037 tablet_service.cc:1467] Processing CreateTablet for tablet 8672d8e5706249ca879c9fafd601867d (DEFAULT_TABLE table=test-table [id=bbf6ca435344455a9a0595f6a3eee168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:48.818485 8178 tablet_service.cc:1467] Processing CreateTablet for tablet 8672d8e5706249ca879c9fafd601867d (DEFAULT_TABLE table=test-table [id=bbf6ca435344455a9a0595f6a3eee168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:48.818616 8037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8672d8e5706249ca879c9fafd601867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:48.818689 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8672d8e5706249ca879c9fafd601867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:48.818692 8307 tablet_service.cc:1467] Processing CreateTablet for tablet 8672d8e5706249ca879c9fafd601867d (DEFAULT_TABLE table=test-table [id=bbf6ca435344455a9a0595f6a3eee168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:48.818897 8307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8672d8e5706249ca879c9fafd601867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:48.820667 8492 tablet_bootstrap.cc:492] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Bootstrap starting.
I20260927 18:37:48.820727 8491 tablet_bootstrap.cc:492] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc: Bootstrap starting.
I20260927 18:37:48.820765 8490 tablet_bootstrap.cc:492] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Bootstrap starting.
I20260927 18:37:48.821336 8492 tablet_bootstrap.cc:654] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:48.821482 8491 tablet_bootstrap.cc:654] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:48.821509 8490 tablet_bootstrap.cc:654] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:48.822391 8492 tablet_bootstrap.cc:492] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: No bootstrap required, opened a new log
I20260927 18:37:48.822467 8492 ts_tablet_manager.cc:1397] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:48.822703 8491 tablet_bootstrap.cc:492] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc: No bootstrap required, opened a new log
I20260927 18:37:48.822703 8490 tablet_bootstrap.cc:492] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: No bootstrap required, opened a new log
I20260927 18:37:48.822773 8490 ts_tablet_manager.cc:1397] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:48.822773 8491 ts_tablet_manager.cc:1397] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:48.822803 8492 raft_consensus.cc:348] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.822875 8492 raft_consensus.cc:374] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:48.822892 8492 raft_consensus.cc:729] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd90722a0b49465cb893df2a27dd8280, State: Initialized, Role: FOLLOWER
I20260927 18:37:48.822984 8492 consensus_queue.cc:260] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.823112 8490 raft_consensus.cc:348] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.823101 8491 raft_consensus.cc:348] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.823194 8490 raft_consensus.cc:374] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:48.823197 8491 raft_consensus.cc:374] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:48.823212 8492 ts_tablet_manager.cc:1428] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:37:48.823225 8490 raft_consensus.cc:729] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23bf25d293a4aa6982e00efa0559ef2, State: Initialized, Role: FOLLOWER
I20260927 18:37:48.823240 8491 raft_consensus.cc:729] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Initialized, Role: FOLLOWER
I20260927 18:37:48.823336 8491 consensus_queue.cc:260] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.823336 8490 consensus_queue.cc:260] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:48.823566 8491 ts_tablet_manager.cc:1428] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:37:48.823566 8490 ts_tablet_manager.cc:1428] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:37:49.132344 8418 raft_consensus.cc:482] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:37:49.132483 8418 raft_consensus.cc:504] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:49.132889 8418 leader_election.cc:288] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:49.133317 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8672d8e5706249ca879c9fafd601867d" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2" is_pre_election: true
I20260927 18:37:49.133322 8197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8672d8e5706249ca879c9fafd601867d" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280" is_pre_election: true
I20260927 18:37:49.133442 8338 raft_consensus.cc:2454] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:49.133476 8197 raft_consensus.cc:2454] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 0.
I20260927 18:37:49.133744 7992 leader_election.cc:302] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, f23bf25d293a4aa6982e00efa0559ef2; no voters:
I20260927 18:37:49.133925 8418 raft_consensus.cc:2781] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:37:49.133976 8418 raft_consensus.cc:482] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:37:49.134013 8418 raft_consensus.cc:3037] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:49.134943 8418 raft_consensus.cc:504] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:49.135246 8418 leader_election.cc:288] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Requested vote from peers fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275), f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039)
I20260927 18:37:49.135566 8197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8672d8e5706249ca879c9fafd601867d" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd90722a0b49465cb893df2a27dd8280"
I20260927 18:37:49.135568 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8672d8e5706249ca879c9fafd601867d" candidate_uuid: "eb4fc38a6b2748699758376ce65801bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23bf25d293a4aa6982e00efa0559ef2"
I20260927 18:37:49.135677 8197 raft_consensus.cc:3037] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:49.135697 8338 raft_consensus.cc:3037] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:49.136616 8197 raft_consensus.cc:2454] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:49.136662 8338 raft_consensus.cc:2454] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb4fc38a6b2748699758376ce65801bc in term 1.
I20260927 18:37:49.136855 7989 leader_election.cc:302] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb4fc38a6b2748699758376ce65801bc, fd90722a0b49465cb893df2a27dd8280; no voters:
I20260927 18:37:49.136997 8418 raft_consensus.cc:2781] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:49.137079 8418 raft_consensus.cc:686] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [term 1 LEADER]: Becoming Leader. State: Replica: eb4fc38a6b2748699758376ce65801bc, State: Running, Role: LEADER
I20260927 18:37:49.137234 8418 consensus_queue.cc:237] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } }
I20260927 18:37:49.138684 7899 catalog_manager.cc:5495] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc reported cstate change: term changed from 0 to 1, leader changed from <none> to eb4fc38a6b2748699758376ce65801bc (127.6.4.1). New cstate: current_term: 1 leader_uuid: "eb4fc38a6b2748699758376ce65801bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb4fc38a6b2748699758376ce65801bc" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 } health_report { overall_health: UNKNOWN } } }
I20260927 18:37:49.189872 8338 raft_consensus.cc:1261] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:49.189895 8197 raft_consensus.cc:1261] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Refusing update from remote peer eb4fc38a6b2748699758376ce65801bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:37:49.190237 8473 consensus_queue.cc:1035] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd90722a0b49465cb893df2a27dd8280" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:37:49.190378 8466 consensus_queue.cc:1035] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23bf25d293a4aa6982e00efa0559ef2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:37:49.201263 8459 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 18:37:50.744340 7899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:50.744475 7899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:37:50.746484 7899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c24d78ebda514fb1b7a8310e47ecea9a: Sending DeleteTablet for 3 replicas of tablet 8672d8e5706249ca879c9fafd601867d
I20260927 18:37:50.746994 8037 tablet_service.cc:1514] Processing DeleteTablet for tablet 8672d8e5706249ca879c9fafd601867d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:50 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:57494
I20260927 18:37:50.747001 8307 tablet_service.cc:1514] Processing DeleteTablet for tablet 8672d8e5706249ca879c9fafd601867d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:50 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:47774
I20260927 18:37:50.747236 8178 tablet_service.cc:1514] Processing DeleteTablet for tablet 8672d8e5706249ca879c9fafd601867d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:37:50 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:44228
I20260927 18:37:50.747506 8530 tablet_replica.cc:330] T 8672d8e5706249ca879c9fafd601867d P eb4fc38a6b2748699758376ce65801bc: stopping tablet replica
I20260927 18:37:50.747515 8531 tablet_replica.cc:330] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: stopping tablet replica
I20260927 18:37:50.747519 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7973
I20260927 18:37:50.749089 8531 raft_consensus.cc:2229] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:50.749140 8529 tablet_replica.cc:330] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: stopping tablet replica
I20260927 18:37:50.749222 8531 raft_consensus.cc:2258] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:50.749235 8529 raft_consensus.cc:2229] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:37:50.749317 8529 raft_consensus.cc:2258] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:37:50.749771 8529 ts_tablet_manager.cc:1905] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:50.749812 8531 ts_tablet_manager.cc:1905] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:37:50.751717 8529 ts_tablet_manager.cc:1918] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:37:50.751717 8531 ts_tablet_manager.cc:1918] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:37:50.751789 8529 log.cc:1196] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/8672d8e5706249ca879c9fafd601867d
I20260927 18:37:50.751789 8531 log.cc:1196] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/8672d8e5706249ca879c9fafd601867d
I20260927 18:37:50.752022 8531 ts_tablet_manager.cc:1939] T 8672d8e5706249ca879c9fafd601867d P fd90722a0b49465cb893df2a27dd8280: Deleting consensus metadata
I20260927 18:37:50.752022 8529 ts_tablet_manager.cc:1939] T 8672d8e5706249ca879c9fafd601867d P f23bf25d293a4aa6982e00efa0559ef2: Deleting consensus metadata
I20260927 18:37:50.752628 7888 catalog_manager.cc:4841] TS f23bf25d293a4aa6982e00efa0559ef2 (127.6.4.3:44039): tablet 8672d8e5706249ca879c9fafd601867d (table test-table [id=bbf6ca435344455a9a0595f6a3eee168]) successfully deleted
I20260927 18:37:50.752714 7885 catalog_manager.cc:4841] TS fd90722a0b49465cb893df2a27dd8280 (127.6.4.2:33275): tablet 8672d8e5706249ca879c9fafd601867d (table test-table [id=bbf6ca435344455a9a0595f6a3eee168]) successfully deleted
W20260927 18:37:50.755966 7888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:39271 (error 108)
I20260927 18:37:50.756186 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 8113
I20260927 18:37:50.757174 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:50.756839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39271 (local address 127.0.0.1:57522)
0927 18:37:50.757039 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:50.757102 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:39271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
W20260927 18:37:50.757300 7888 catalog_manager.cc:4584] TS eb4fc38a6b2748699758376ce65801bc (127.6.4.1:39271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8672d8e5706249ca879c9fafd601867d: Network error: Client connection negotiation failed: client connection to 127.6.4.1:39271: connect: Connection refused (error 111)
I20260927 18:37:50.761356 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 8253
I20260927 18:37:50.765595 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 7869
2026-09-27T18:37:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6292 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:50 dist-test-slave-bzzj krb5kdc[8536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:50 dist-test-slave-bzzj krb5kdc[8536](info): set up 2 sockets
Sep 27 18:37:50 dist-test-slave-bzzj krb5kdc[8536](info): commencing operation
krb5kdc: starting...
W20260927 18:37:51.715628 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:51 dist-test-slave-bzzj krb5kdc[8536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:51.824815 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33239
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:33239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:51.925433 8549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:51.925650 8549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:51.925686 8549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:51.929531 8549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:51.929579 8549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:51.929596 8549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:51.929610 8549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:51.929625 8549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:51.934538 8549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:33239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:51.935395 8549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:51.936292 8549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:51.944345 8554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:51.944439 8555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:51.944489 8557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:51.944360 8549 server_base.cc:1034] running on GCE node
I20260927 18:37:51.945050 8549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:51.946038 8549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:51.947189 8549 hybrid_clock.cc:648] HybridClock initialized: now 1790534271947174 us; error 27 us; skew 500 ppm
Sep 27 18:37:51 dist-test-slave-bzzj krb5kdc[8536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534271, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:51.949694 8549 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:51.950464 8549 webserver.cc:460] Webserver started at http://127.6.4.62:40327/ using document root <none> and password file <none>
I20260927 18:37:51.950940 8549 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:51.951000 8549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:51.951176 8549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:51.953030 8549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64"
format_stamp: "Formatted at 2026-09-27 18:37:51 on dist-test-slave-bzzj"
server_key: "19c26d55fcde15999675067b9f09da91"
server_key_iv: "bd074fd031d76db5b3d0f296fbfea865"
server_key_version: "encryptionkey@0"
I20260927 18:37:51.953521 8549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64"
format_stamp: "Formatted at 2026-09-27 18:37:51 on dist-test-slave-bzzj"
server_key: "19c26d55fcde15999675067b9f09da91"
server_key_iv: "bd074fd031d76db5b3d0f296fbfea865"
server_key_version: "encryptionkey@0"
I20260927 18:37:51.956796 8549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:37:51.958848 8564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:51.959896 8549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:51.959998 8549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64"
format_stamp: "Formatted at 2026-09-27 18:37:51 on dist-test-slave-bzzj"
server_key: "19c26d55fcde15999675067b9f09da91"
server_key_iv: "bd074fd031d76db5b3d0f296fbfea865"
server_key_version: "encryptionkey@0"
I20260927 18:37:51.960078 8549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:51.975814 8549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:51.978186 8549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:51.978348 8549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:51.984375 8549 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:33239
I20260927 18:37:51.984390 8616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:33239 every 8 connection(s)
I20260927 18:37:51.985149 8549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:51.987555 8617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:51.989306 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 8549
I20260927 18:37:51.989416 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:51.989694 6160 external_mini_cluster.cc:1428] Setting key 33e8477fd6f43fb3bc5f2c51b523f0bb
I20260927 18:37:51.993484 8617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64: Bootstrap starting.
I20260927 18:37:51.995744 8617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:51.996579 8617 log.cc:824] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:37:51 dist-test-slave-bzzj krb5kdc[8536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:51.998243 8617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64: No bootstrap required, opened a new log
I20260927 18:37:52.001801 8617 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33239 } }
I20260927 18:37:52.002007 8617 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:52.002048 8617 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd354b5fbd8a4aedbe4fd2a4b1b22e64, State: Initialized, Role: FOLLOWER
I20260927 18:37:52.002313 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:51.990854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.6.4.62:33239)
0927 18:37:51.991194 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:51.991215 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:51.991246 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:37:51.991824 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:51.991839 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:51.992157 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:51.992416 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:51.993206 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:51.994134 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:51.995225 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:51.995639 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:51.995746 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:51.998144 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:51.998149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:51.998160 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:51.998183 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:51.999621 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:51.999999 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.000004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.000008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.000152 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.000445 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.000447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.000448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.000860 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:52.001028 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:52.001286 (+ 258us) server_negotiation.cc:299] Negotiation successful
0927 18:37:52.001536 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":91,"threads_started":1}
I20260927 18:37:52.002439 8617 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33239 } }
I20260927 18:37:52.002568 8617 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:52.002628 8617 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:52.002717 8617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:52.003585 8617 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33239 } }
I20260927 18:37:52.003909 8617 leader_election.cc:302] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd354b5fbd8a4aedbe4fd2a4b1b22e64; no voters:
I20260927 18:37:52.004218 8617 leader_election.cc:288] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:52.004323 8622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:52.004508 8622 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [term 1 LEADER]: Becoming Leader. State: Replica: bd354b5fbd8a4aedbe4fd2a4b1b22e64, State: Running, Role: LEADER
I20260927 18:37:52.004856 8622 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33239 } }
I20260927 18:37:52.005244 8617 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:52.006327 8624 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd354b5fbd8a4aedbe4fd2a4b1b22e64. Latest consensus state: current_term: 1 leader_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33239 } } }
I20260927 18:37:52.006325 8623 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd354b5fbd8a4aedbe4fd2a4b1b22e64" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33239 } } }
I20260927 18:37:52.006457 8624 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:52.006457 8623 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:52.006866 8632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:52.009351 8632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:52.014297 8632 catalog_manager.cc:1347] Generated new cluster ID: b931e679f99248128e9ee10c50b32919
I20260927 18:37:52.014356 8632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:52.026048 8632 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:52.026875 8632 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:52.033900 8632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd354b5fbd8a4aedbe4fd2a4b1b22e64: Generated new TSK 0
I20260927 18:37:52.034430 8632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:52.067039 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33239
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:52.169632 8643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:52.169864 8643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:52.169902 8643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:52.173836 8643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:52.173893 8643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:52.173946 8643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:52.178939 8643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:52.179934 8643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:52.180950 8643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:52.189991 8648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:52.189961 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:52.190100 8643 server_base.cc:1034] running on GCE node
W20260927 18:37:52.189970 8651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:52.190591 8643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:52.191179 8643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:52.192332 8643 hybrid_clock.cc:648] HybridClock initialized: now 1790534272192302 us; error 39 us; skew 500 ppm
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[8536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:52.195173 8643 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:52.196175 8643 webserver.cc:460] Webserver started at http://127.6.4.1:39725/ using document root <none> and password file <none>
I20260927 18:37:52.196712 8643 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:52.196801 8643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:52.197019 8643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:52.198982 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "fae248fa02784729b2b79803be355fd2"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "f782a60e77dcfcf61a3873c25b654512"
server_key_iv: "8f65553fc311efb1a73ed4a77cc41372"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.199498 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "fae248fa02784729b2b79803be355fd2"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "f782a60e77dcfcf61a3873c25b654512"
server_key_iv: "8f65553fc311efb1a73ed4a77cc41372"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.203095 8643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:37:52.205446 8658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.206653 8643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:52.206753 8643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "fae248fa02784729b2b79803be355fd2"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "f782a60e77dcfcf61a3873c25b654512"
server_key_iv: "8f65553fc311efb1a73ed4a77cc41372"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.206843 8643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:52.245811 8643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:52.248574 8643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:52.248744 8643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:52.249293 8643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:52.250069 8643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:52.250115 8643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.250168 8643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:52.250193 8643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.258116 8643 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:42931
I20260927 18:37:52.258147 8771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:42931 every 8 connection(s)
I20260927 18:37:52.258941 8643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:37:52.262418 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 8643
I20260927 18:37:52.262530 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:52.262845 6160 external_mini_cluster.cc:1428] Setting key dda88c245df6d6dc301259e8714f6f38
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[8536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:37:52.273195 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:52.260599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:48793 (local address 127.6.4.62:33239)
0927 18:37:52.260716 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:52.260720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:52.261503 (+ 783us) server_negotiation.cc:407] Connection header received
0927 18:37:52.263070 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:52.263073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:52.263123 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:52.263194 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:52.264597 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.265117 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.266463 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.266695 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.266783 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:52.269450 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:52.269454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:52.269457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:52.269487 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:52.271016 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.271520 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.271525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.271526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.271611 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.271955 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.271958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.271960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.272191 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:52.272273 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:52.272950 (+ 677us) server_negotiation.cc:299] Negotiation successful
0927 18:37:52.273070 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:37:52.274046 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:52.260944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33239 (local address 127.6.4.1:48793)
0927 18:37:52.261353 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:52.261392 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:52.262850 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:52.263372 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:52.263380 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:52.263817 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:52.264441 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:52.264463 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.265241 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.265244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:52.266342 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:52.266352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.266777 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.266832 (+ 55us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:52.266993 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:52.267610 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:52.267627 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:52.269275 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:52.271149 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:52.271154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:52.271166 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:52.271419 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:52.271700 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:52.271702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:52.271704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:52.271869 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:52.272295 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:52.272301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:52.272602 (+ 301us) client_negotiation.cc:769] Sending connection context
0927 18:37:52.272886 (+ 284us) client_negotiation.cc:240] Negotiation successful
0927 18:37:52.273162 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:37:52.275213 8772 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33239
I20260927 18:37:52.275439 8772 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:52.275892 8772 heartbeater.cc:511] Master 127.6.4.62:33239 requested a full tablet report, sending...
I20260927 18:37:52.277391 8581 ts_manager.cc:194] Registered new tserver with Master: fae248fa02784729b2b79803be355fd2 (127.6.4.1:42931)
I20260927 18:37:52.278491 8581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:48793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:52.290432 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33239
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:52.396828 8777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:52.397063 8777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:52.397119 8777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:52.401010 8777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:52.401063 8777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:52.401139 8777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:52.406927 8777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:52.407848 8777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:52.408897 8777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:52.418375 8783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:52.418381 8782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:52.418380 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:52.418598 8777 server_base.cc:1034] running on GCE node
I20260927 18:37:52.418984 8777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:52.419536 8777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:52.420689 8777 hybrid_clock.cc:648] HybridClock initialized: now 1790534272420673 us; error 31 us; skew 500 ppm
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[8536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:52.423099 8777 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:52.423851 8777 webserver.cc:460] Webserver started at http://127.6.4.2:36997/ using document root <none> and password file <none>
I20260927 18:37:52.424357 8777 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:52.424414 8777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:52.424562 8777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:52.426369 8777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "ac97101f9b8240949538d8b14e3eb472"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "8698a146120416f4b04c93af0474f900"
server_key_iv: "1551bd1805446a07e85c85fd41859d5f"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.426837 8777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "ac97101f9b8240949538d8b14e3eb472"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "8698a146120416f4b04c93af0474f900"
server_key_iv: "1551bd1805446a07e85c85fd41859d5f"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.430099 8777 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:52.432296 8792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.433379 8777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:52.433467 8777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "ac97101f9b8240949538d8b14e3eb472"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "8698a146120416f4b04c93af0474f900"
server_key_iv: "1551bd1805446a07e85c85fd41859d5f"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.433553 8777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:52.465171 8777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:52.467782 8777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:52.467914 8777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:52.468446 8777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:52.469172 8777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:52.469213 8777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.469249 8777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:52.469269 8777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.476981 8777 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36921
I20260927 18:37:52.477005 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36921 every 8 connection(s)
I20260927 18:37:52.477766 8777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:52.485862 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 8777
I20260927 18:37:52.485980 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:52.486244 6160 external_mini_cluster.cc:1428] Setting key acb28b6c382e3cde9a66b9852e5ed32a
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[8536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:52.490444 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:52.479456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:60715 (local address 127.6.4.62:33239)
0927 18:37:52.479600 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:52.479604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:52.480458 (+ 854us) server_negotiation.cc:407] Connection header received
0927 18:37:52.481799 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:52.481802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:52.481847 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:52.481917 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:52.482940 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.483475 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.484633 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.484848 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.484966 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:52.487179 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:52.487182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:52.487184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:52.487207 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:52.488658 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.489076 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.489079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.489080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.489180 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.489514 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.489516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.489517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.489678 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:52.489749 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:52.490250 (+ 501us) server_negotiation.cc:299] Negotiation successful
0927 18:37:52.490336 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:37:52.491209 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:52.479826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33239 (local address 127.6.4.2:60715)
0927 18:37:52.480314 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:52.480361 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:52.481598 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:52.482054 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:52.482061 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:52.482376 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:52.482836 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:52.482848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.483580 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.483582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:52.484499 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:52.484507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.484976 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.485026 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:52.485176 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:52.485628 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:52.485643 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:52.487037 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:52.488754 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:52.488759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:52.488770 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:52.488997 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:52.489270 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:52.489272 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:52.489274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:52.489434 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:52.489762 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:52.489767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:52.490032 (+ 265us) client_negotiation.cc:769] Sending connection context
0927 18:37:52.490220 (+ 188us) client_negotiation.cc:240] Negotiation successful
0927 18:37:52.490479 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":95,"threads_started":1}
I20260927 18:37:52.492255 8906 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33239
I20260927 18:37:52.492473 8906 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:52.492986 8906 heartbeater.cc:511] Master 127.6.4.62:33239 requested a full tablet report, sending...
I20260927 18:37:52.494103 8581 ts_manager.cc:194] Registered new tserver with Master: ac97101f9b8240949538d8b14e3eb472 (127.6.4.2:36921)
I20260927 18:37:52.494830 8581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:60715
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:52.512264 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33239
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:52.619272 8911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:52.619481 8911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:52.619520 8911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:52.623438 8911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:52.623487 8911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:52.623548 8911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:52.628603 8911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:52.629463 8911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:52.630348 8911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:52.638996 8919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:52.639055 8916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:52.639132 8917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:52.639216 8911 server_base.cc:1034] running on GCE node
I20260927 18:37:52.639534 8911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:52.640074 8911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:52.641237 8911 hybrid_clock.cc:648] HybridClock initialized: now 1790534272641213 us; error 37 us; skew 500 ppm
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[8536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:52.643599 8911 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:52.644408 8911 webserver.cc:460] Webserver started at http://127.6.4.3:34939/ using document root <none> and password file <none>
I20260927 18:37:52.644896 8911 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:52.644954 8911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:52.645121 8911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:52.646958 8911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "0c20f86574644821aced3c30d972f96c"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "c60977798184c291a0e8b8a23aa0c8a3"
server_key_iv: "0219988c6fb657c786f3f28221dbed9f"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.647465 8911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "0c20f86574644821aced3c30d972f96c"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "c60977798184c291a0e8b8a23aa0c8a3"
server_key_iv: "0219988c6fb657c786f3f28221dbed9f"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.650741 8911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:37:52.652835 8926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.653931 8911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:52.654026 8911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "0c20f86574644821aced3c30d972f96c"
format_stamp: "Formatted at 2026-09-27 18:37:52 on dist-test-slave-bzzj"
server_key: "c60977798184c291a0e8b8a23aa0c8a3"
server_key_iv: "0219988c6fb657c786f3f28221dbed9f"
server_key_version: "encryptionkey@0"
I20260927 18:37:52.654105 8911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:52.666654 8911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:52.669029 8911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:52.669178 8911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:52.669663 8911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:52.670387 8911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:52.670429 8911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.670476 8911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:52.670504 8911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:52.678191 8911 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:42199
I20260927 18:37:52.678215 9039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:42199 every 8 connection(s)
I20260927 18:37:52.678934 8911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[8536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:52.687129 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 8911
I20260927 18:37:52.687225 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:52.687490 6160 external_mini_cluster.cc:1428] Setting key ec235d53abaee8bb8ac29288108ae289
I20260927 18:37:52.690680 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:52.680444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:60787 (local address 127.6.4.62:33239)
0927 18:37:52.680551 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:52.680554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:52.681257 (+ 703us) server_negotiation.cc:407] Connection header received
0927 18:37:52.682503 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:52.682506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:52.682564 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:52.682641 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:52.683671 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.684226 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.685300 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.685516 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.685601 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:52.687707 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:52.687711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:52.687713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:52.687739 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:52.689056 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.689556 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.689559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.689560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.689636 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:52.689880 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:52.689882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:52.689883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:52.690028 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:52.690093 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:52.690462 (+ 369us) server_negotiation.cc:299] Negotiation successful
0927 18:37:52.690542 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:37:52.691375 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:52.680743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33239 (local address 127.6.4.3:60787)
0927 18:37:52.681130 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:52.681167 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:52.682313 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:52.682798 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:52.682805 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:52.683106 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:52.683530 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:52.683542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.684339 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.684341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:52.685183 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:52.685192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:52.685596 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:52.685644 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:52.685793 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:52.686245 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:52.686259 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:52.687550 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:52.689160 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:52.689165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:52.689177 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:52.689479 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:52.689701 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:52.689704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:52.689706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:52.689821 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:52.690092 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:52.690096 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:52.690297 (+ 201us) client_negotiation.cc:769] Sending connection context
0927 18:37:52.690432 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 18:37:52.690645 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
I20260927 18:37:52.692274 9040 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33239
I20260927 18:37:52.692456 9040 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:52.692831 9040 heartbeater.cc:511] Master 127.6.4.62:33239 requested a full tablet report, sending...
I20260927 18:37:52.693676 8581 ts_manager.cc:194] Registered new tserver with Master: 0c20f86574644821aced3c30d972f96c (127.6.4.3:42199)
I20260927 18:37:52.694198 8581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:60787
I20260927 18:37:52.700630 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:37:52.711858 8620 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:37:52.710660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.6.4.62:33239)
0927 18:37:52.710796 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:52.710799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:52.710871 (+ 72us) server_negotiation.cc:407] Connection header received
0927 18:37:52.710971 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:52.710973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:52.711027 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:52.711096 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:52.711740 (+ 644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38404: BlockingRecv error: recv got EOF from 127.0.0.1:38404 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260927 18:37:52.713431 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 8643
I20260927 18:37:52.717945 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 8777
I20260927 18:37:52.721979 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 8911
I20260927 18:37:52.726042 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 8549
2026-09-27T18:37:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1959 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[9052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[9052](info): set up 2 sockets
Sep 27 18:37:52 dist-test-slave-bzzj krb5kdc[9052](info): commencing operation
krb5kdc: starting...
W20260927 18:37:53.675736 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:53 dist-test-slave-bzzj krb5kdc[9052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:53.786599 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33109
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:53.888684 9065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:53.888923 9065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:53.888960 9065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:53.892858 9065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:53.892901 9065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:53.892918 9065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:53.892933 9065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:53.892947 9065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:53.897914 9065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:53.898801 9065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:53.899714 9065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:53.907461 9070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:53.907467 9071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:53.907635 9065 server_base.cc:1034] running on GCE node
W20260927 18:37:53.907505 9073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:53.908289 9065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:53.909272 9065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:53.910423 9065 hybrid_clock.cc:648] HybridClock initialized: now 1790534273910398 us; error 35 us; skew 500 ppm
Sep 27 18:37:53 dist-test-slave-bzzj krb5kdc[9052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534273, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:53.912988 9065 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:53.913806 9065 webserver.cc:460] Webserver started at http://127.6.4.62:35775/ using document root <none> and password file <none>
I20260927 18:37:53.914314 9065 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:53.914372 9065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:53.914556 9065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:53.916469 9065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce"
format_stamp: "Formatted at 2026-09-27 18:37:53 on dist-test-slave-bzzj"
server_key: "e92c5d7afcb29a485c37d1b99ff242cf"
server_key_iv: "d9ea6ae503b6f4c1d2bcd81b6207a848"
server_key_version: "encryptionkey@0"
I20260927 18:37:53.916978 9065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce"
format_stamp: "Formatted at 2026-09-27 18:37:53 on dist-test-slave-bzzj"
server_key: "e92c5d7afcb29a485c37d1b99ff242cf"
server_key_iv: "d9ea6ae503b6f4c1d2bcd81b6207a848"
server_key_version: "encryptionkey@0"
I20260927 18:37:53.920365 9065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:37:53.922580 9080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:53.923705 9065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:53.923803 9065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce"
format_stamp: "Formatted at 2026-09-27 18:37:53 on dist-test-slave-bzzj"
server_key: "e92c5d7afcb29a485c37d1b99ff242cf"
server_key_iv: "d9ea6ae503b6f4c1d2bcd81b6207a848"
server_key_version: "encryptionkey@0"
I20260927 18:37:53.923892 9065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:53.947963 9065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:53.950634 9065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:53.950799 9065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:53.957207 9065 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:33109
I20260927 18:37:53.957443 9132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:33109 every 8 connection(s)
I20260927 18:37:53.958096 9065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:53.960672 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:53.961375 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9065
I20260927 18:37:53.961486 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:53.961773 6160 external_mini_cluster.cc:1428] Setting key c3067750d698b062761dfb93b5d868e5
I20260927 18:37:53.966440 9133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce: Bootstrap starting.
I20260927 18:37:53.968573 9133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:53.969239 9133 log.cc:824] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:37:53 dist-test-slave-bzzj krb5kdc[9052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:53.971189 9133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce: No bootstrap required, opened a new log
I20260927 18:37:53.974666 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:53.962992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.6.4.62:33109)
0927 18:37:53.963383 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:53.963397 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:53.963430 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:37:53.964141 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:53.964159 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:53.964481 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:53.964779 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:53.965538 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:53.966487 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:53.967312 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:53.967761 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:53.967907 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:53.970389 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:53.970394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:53.970406 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:53.970430 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:53.972011 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:53.972493 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:53.972497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:53.972501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:53.972634 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:53.972890 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:53.972891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:53.972893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:53.973240 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:53.973378 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:53.973693 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 18:37:53.973964 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":117,"threads_started":1}
I20260927 18:37:53.974854 9133 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33109 } }
I20260927 18:37:53.975066 9133 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:53.975121 9133 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f7dc36e24cd8a3980af27c4e5dce, State: Initialized, Role: FOLLOWER
I20260927 18:37:53.975509 9133 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33109 } }
I20260927 18:37:53.975652 9133 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:53.975719 9133 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:53.975808 9133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:53.976768 9133 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33109 } }
I20260927 18:37:53.977084 9133 leader_election.cc:302] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7e4f7dc36e24cd8a3980af27c4e5dce; no voters:
I20260927 18:37:53.977378 9133 leader_election.cc:288] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:53.977450 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:37:53.977451 9138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:53.977631 9138 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [term 1 LEADER]: Becoming Leader. State: Replica: a7e4f7dc36e24cd8a3980af27c4e5dce, State: Running, Role: LEADER
I20260927 18:37:53.977902 9138 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33109 } }
I20260927 18:37:53.978560 9133 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:53.978973 9138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33109 } } }
I20260927 18:37:53.979067 9138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:53.979365 9139 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7e4f7dc36e24cd8a3980af27c4e5dce. Latest consensus state: current_term: 1 leader_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7e4f7dc36e24cd8a3980af27c4e5dce" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33109 } } }
I20260927 18:37:53.979470 9139 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce [sys.catalog]: This master's current role is: LEADER
W20260927 18:37:53.983013 9154 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:37:53.983086 9154 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:37:53.983161 9143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:53.983958 9143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:53.989264 9143 catalog_manager.cc:1347] Generated new cluster ID: 61cdeb29e64640e48342b75b2528c8f6
I20260927 18:37:53.989320 9143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:53.999462 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:53.995187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58572 (local address 127.6.4.62:33109)
0927 18:37:53.995303 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:53.995307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:53.995529 (+ 222us) server_negotiation.cc:407] Connection header received
0927 18:37:53.995643 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:53.995646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:53.995691 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:53.995758 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:53.996275 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:53.996808 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:53.997607 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:53.997848 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:53.997930 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:53.998657 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:53.998661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:53.998664 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:37:53.999061 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:53.999088 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:37:53.999103 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:53.999175 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:53.999282 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 18:37:53.999336 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:37:54.001766 9143 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:54.002899 9143 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:54.016739 9143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7e4f7dc36e24cd8a3980af27c4e5dce: Generated new TSK 0
I20260927 18:37:54.017400 9143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:37:54.029471 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9065
2026-09-27T18:37:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1302 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:54 dist-test-slave-bzzj krb5kdc[9166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:54 dist-test-slave-bzzj krb5kdc[9166](info): set up 2 sockets
Sep 27 18:37:54 dist-test-slave-bzzj krb5kdc[9166](info): commencing operation
krb5kdc: starting...
W20260927 18:37:54.975731 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:55.087121 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:41277
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:38145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:41277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:55.190843 9179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:55.191092 9179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:55.191145 9179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:55.195083 9179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:55.195137 9179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:55.195161 9179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:55.195181 9179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:55.195206 9179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:55.200222 9179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:38145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:41277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:41277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:55.201153 9179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:55.202087 9179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:55.210697 9185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.210716 9187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.210706 9184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:55.210919 9179 server_base.cc:1034] running on GCE node
I20260927 18:37:55.211472 9179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:55.212605 9179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:55.213747 9179 hybrid_clock.cc:648] HybridClock initialized: now 1790534275213722 us; error 39 us; skew 500 ppm
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:55.216337 9179 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:55.217181 9179 webserver.cc:460] Webserver started at http://127.6.4.62:35255/ using document root <none> and password file <none>
I20260927 18:37:55.217689 9179 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:55.217746 9179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:55.217929 9179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:55.219904 9179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "890ad6f8055440ccaa82c1f521226de6"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "3a53e52d5c6c9aca294d5f12f0c01546"
server_key_iv: "2049def797ddba634c69b233c3663d89"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.220492 9179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "890ad6f8055440ccaa82c1f521226de6"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "3a53e52d5c6c9aca294d5f12f0c01546"
server_key_iv: "2049def797ddba634c69b233c3663d89"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.223917 9179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:55.226159 9194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.227313 9179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:55.227420 9179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "890ad6f8055440ccaa82c1f521226de6"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "3a53e52d5c6c9aca294d5f12f0c01546"
server_key_iv: "2049def797ddba634c69b233c3663d89"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.227504 9179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:55.241447 9179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:55.243942 9179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:55.244139 9179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:55.250806 9179 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:41277
I20260927 18:37:55.251024 9246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:41277 every 8 connection(s)
I20260927 18:37:55.251740 9179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:55.252234 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9179
I20260927 18:37:55.252339 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:55.252839 6160 external_mini_cluster.cc:1428] Setting key 1079cf077646b0e003677538daea3f6c
I20260927 18:37:55.255273 9247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:55.262192 9247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6: Bootstrap starting.
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:55.264946 9247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:55.265833 9247 log.cc:824] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:55.268318 9247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6: No bootstrap required, opened a new log
I20260927 18:37:55.269490 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.254282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.6.4.62:41277)
0927 18:37:55.254768 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:55.254787 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:55.254832 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:37:55.255728 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:55.255755 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:55.256254 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:55.256640 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:55.257970 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.259501 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.260633 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.261315 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.261851 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.264117 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:55.264124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:55.264141 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:55.264173 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:55.266286 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.266805 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.266811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.266817 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.266952 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.267229 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.267232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.267234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.267731 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:55.267918 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:55.268247 (+ 329us) server_negotiation.cc:299] Negotiation successful
0927 18:37:55.268547 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":136,"threads_started":1}
I20260927 18:37:55.272619 9247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ad6f8055440ccaa82c1f521226de6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41277 } }
I20260927 18:37:55.272898 9247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:55.272959 9247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890ad6f8055440ccaa82c1f521226de6, State: Initialized, Role: FOLLOWER
I20260927 18:37:55.273418 9247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ad6f8055440ccaa82c1f521226de6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41277 } }
I20260927 18:37:55.273597 9247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:55.273658 9247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:55.273767 9247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:55.274837 9247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ad6f8055440ccaa82c1f521226de6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41277 } }
I20260927 18:37:55.275197 9247 leader_election.cc:302] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 890ad6f8055440ccaa82c1f521226de6; no voters:
I20260927 18:37:55.275527 9247 leader_election.cc:288] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:55.275638 9252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:55.275832 9252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [term 1 LEADER]: Becoming Leader. State: Replica: 890ad6f8055440ccaa82c1f521226de6, State: Running, Role: LEADER
I20260927 18:37:55.276190 9252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ad6f8055440ccaa82c1f521226de6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41277 } }
I20260927 18:37:55.276803 9247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:55.278079 9254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 890ad6f8055440ccaa82c1f521226de6. Latest consensus state: current_term: 1 leader_uuid: "890ad6f8055440ccaa82c1f521226de6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ad6f8055440ccaa82c1f521226de6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41277 } } }
I20260927 18:37:55.278179 9254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:55.278512 9262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:55.278884 9253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "890ad6f8055440ccaa82c1f521226de6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ad6f8055440ccaa82c1f521226de6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41277 } } }
I20260927 18:37:55.278976 9253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:55.281873 9262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:55.287736 9262 catalog_manager.cc:1347] Generated new cluster ID: dbee0c89b10f4260b056d80b8b1c548f
I20260927 18:37:55.287802 9262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:55.304026 9262 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:55.305279 9262 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:55.318081 9262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 890ad6f8055440ccaa82c1f521226de6: Generated new TSK 0
I20260927 18:37:55.318871 9262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:55.344612 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:41277
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:38145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:55.446720 9273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:55.446938 9273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:55.446990 9273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:55.451007 9273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:55.451057 9273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:55.451133 9273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:55.455977 9273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:38145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:41277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:55.456852 9273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:55.457729 9273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:55.466156 9281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.466151 9279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.466151 9278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:55.466352 9273 server_base.cc:1034] running on GCE node
I20260927 18:37:55.466813 9273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:55.467357 9273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:55.468502 9273 hybrid_clock.cc:648] HybridClock initialized: now 1790534275468475 us; error 40 us; skew 500 ppm
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:55.470927 9273 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:55.471657 9273 webserver.cc:460] Webserver started at http://127.6.4.1:45383/ using document root <none> and password file <none>
I20260927 18:37:55.472162 9273 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:55.472227 9273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:55.472400 9273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:55.474210 9273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "b8d231c2e7ee43b197009c03ecaf6590"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "43abd255b072484da1372b12039e306e"
server_key_iv: "4c5198a0811da19c3104b95187e838a4"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.474706 9273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "b8d231c2e7ee43b197009c03ecaf6590"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "43abd255b072484da1372b12039e306e"
server_key_iv: "4c5198a0811da19c3104b95187e838a4"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.478075 9273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:55.480309 9288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.481428 9273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:37:55.481534 9273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "b8d231c2e7ee43b197009c03ecaf6590"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "43abd255b072484da1372b12039e306e"
server_key_iv: "4c5198a0811da19c3104b95187e838a4"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.481614 9273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:55.509328 9273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:55.512024 9273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:55.512203 9273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:55.512727 9273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:55.513476 9273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:55.513520 9273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.513581 9273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:55.513610 9273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.521620 9273 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:34639
I20260927 18:37:55.521657 9401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:34639 every 8 connection(s)
I20260927 18:37:55.522401 9273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:37:55.529862 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9273
I20260927 18:37:55.529954 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:55.530222 6160 external_mini_cluster.cc:1428] Setting key 6981f87f9a5862678b1d013829b41a44
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:55.534515 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.524040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:58657 (local address 127.6.4.62:41277)
0927 18:37:55.524178 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:55.524181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:55.524928 (+ 747us) server_negotiation.cc:407] Connection header received
0927 18:37:55.526177 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:55.526180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:55.526224 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:55.526292 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:55.527272 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.527789 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.528885 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.529127 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.529215 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.531450 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:55.531454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:55.531457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:55.531481 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:55.532703 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.533138 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.533142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.533143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.533227 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.533516 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.533519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.533521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.533704 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:55.533789 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:55.534246 (+ 457us) server_negotiation.cc:299] Negotiation successful
0927 18:37:55.534365 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:37:55.535262 9404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.524382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:41277 (local address 127.6.4.1:58657)
0927 18:37:55.524794 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:55.524832 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:55.525996 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:55.526415 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:55.526421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:55.526729 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:55.527164 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:55.527179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.527872 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.527874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:55.528765 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:55.528774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.529208 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.529244 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:55.529365 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.529837 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:55.529851 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:55.531297 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:55.532806 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:55.532811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:55.532825 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:55.533049 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:55.533305 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:55.533308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:55.533310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:55.533433 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:55.533798 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:55.533804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:55.534029 (+ 225us) client_negotiation.cc:769] Sending connection context
0927 18:37:55.534183 (+ 154us) client_negotiation.cc:240] Negotiation successful
0927 18:37:55.534467 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":109,"threads_started":1}
I20260927 18:37:55.536367 9402 heartbeater.cc:347] Connected to a master server at 127.6.4.62:41277
I20260927 18:37:55.536609 9402 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:55.537133 9402 heartbeater.cc:511] Master 127.6.4.62:41277 requested a full tablet report, sending...
I20260927 18:37:55.538733 9210 ts_manager.cc:194] Registered new tserver with Master: b8d231c2e7ee43b197009c03ecaf6590 (127.6.4.1:34639)
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:37:55.539898 9210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:58657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:55.554216 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:41277
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:38145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:55.657048 9407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:55.657269 9407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:55.657306 9407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:55.661106 9407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:55.661147 9407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:55.661198 9407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:55.666002 9407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:38145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:41277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:55.666837 9407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:55.667675 9407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:55.676417 9415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:55.676498 9407 server_base.cc:1034] running on GCE node
W20260927 18:37:55.676429 9413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.676429 9412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:55.677007 9407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:55.677603 9407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:55.678745 9407 hybrid_clock.cc:648] HybridClock initialized: now 1790534275678729 us; error 26 us; skew 500 ppm
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:55.681174 9407 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:55.681936 9407 webserver.cc:460] Webserver started at http://127.6.4.2:44335/ using document root <none> and password file <none>
I20260927 18:37:55.682415 9407 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:55.682475 9407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:55.682638 9407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:55.684480 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "f75cdfe4cbec40ef9efdf45780018919"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "574b9f3f3ea0ef97975197897fb27efe"
server_key_iv: "d80d233d5760bd35179d4282007f3fad"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.685027 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "f75cdfe4cbec40ef9efdf45780018919"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "574b9f3f3ea0ef97975197897fb27efe"
server_key_iv: "d80d233d5760bd35179d4282007f3fad"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.688350 9407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 18:37:55.690460 9422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.691562 9407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:55.691660 9407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "f75cdfe4cbec40ef9efdf45780018919"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "574b9f3f3ea0ef97975197897fb27efe"
server_key_iv: "d80d233d5760bd35179d4282007f3fad"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.691749 9407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:55.702099 9407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:55.704566 9407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:55.704735 9407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:55.705214 9407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:55.705922 9407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:55.705967 9407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.706015 9407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:55.706043 9407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.713619 9407 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:41611
I20260927 18:37:55.713650 9535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:41611 every 8 connection(s)
I20260927 18:37:55.714376 9407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:55.718892 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9407
I20260927 18:37:55.719004 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:55.719283 6160 external_mini_cluster.cc:1428] Setting key 7d61b515148ac5bdbd7bbda3559854d4
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:55.727048 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.715915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51537 (local address 127.6.4.62:41277)
0927 18:37:55.716040 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:55.716044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:55.717249 (+ 1205us) server_negotiation.cc:407] Connection header received
0927 18:37:55.718055 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:55.718058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:55.718113 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:55.718184 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:55.719438 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.720046 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.721464 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.721677 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.721759 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.724002 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:55.724005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:55.724006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:55.724029 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:55.725311 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.725739 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.725742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.725744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.725814 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.726091 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.726094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.726096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.726257 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:55.726338 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:55.726785 (+ 447us) server_negotiation.cc:299] Negotiation successful
0927 18:37:55.726898 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
I20260927 18:37:55.727746 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.716323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:41277 (local address 127.6.4.2:51537)
0927 18:37:55.717113 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:55.717159 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:55.717872 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:55.718344 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:55.718350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:55.718658 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:55.719270 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:55.719289 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.720180 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.720183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:55.721357 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:55.721368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.721763 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.721808 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:55.721984 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.722570 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:55.722584 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:55.723859 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:55.725407 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:55.725412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:55.725425 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:55.725653 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:55.725889 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:55.725892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:55.725894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:55.726012 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:55.726347 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:55.726353 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:55.726572 (+ 219us) client_negotiation.cc:769] Sending connection context
0927 18:37:55.726728 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 18:37:55.727007 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":497,"threads_started":1}
I20260927 18:37:55.728732 9536 heartbeater.cc:347] Connected to a master server at 127.6.4.62:41277
I20260927 18:37:55.728930 9536 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:55.729370 9536 heartbeater.cc:511] Master 127.6.4.62:41277 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:37:55.730280 9210 ts_manager.cc:194] Registered new tserver with Master: f75cdfe4cbec40ef9efdf45780018919 (127.6.4.2:41611)
I20260927 18:37:55.730835 9210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:51537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:55.742540 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:41277
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:38145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:55.845294 9541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:55.845523 9541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:55.845562 9541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:55.849535 9541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:55.849589 9541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:55.849644 9541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:55.854548 9541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:38145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:41277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:55.855417 9541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:55.856319 9541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:55.864851 9546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.864862 9547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:55.864898 9549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:55.865181 9541 server_base.cc:1034] running on GCE node
I20260927 18:37:55.865499 9541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:55.866016 9541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:55.867170 9541 hybrid_clock.cc:648] HybridClock initialized: now 1790534275867149 us; error 32 us; skew 500 ppm
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:55.875937 9541 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:55.876775 9541 webserver.cc:460] Webserver started at http://127.6.4.3:42547/ using document root <none> and password file <none>
I20260927 18:37:55.877290 9541 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:55.877364 9541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:55.877560 9541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:55.879528 9541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "d156d815c0ca412bae6ea83aa1f1f927"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "2b84b071b02cb61f732a85c994fa0f30"
server_key_iv: "9dbf22d565d788529cf3ed23d360646a"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.880043 9541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "d156d815c0ca412bae6ea83aa1f1f927"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "2b84b071b02cb61f732a85c994fa0f30"
server_key_iv: "9dbf22d565d788529cf3ed23d360646a"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.883282 9541 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:37:55.885505 9556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.886590 9541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:37:55.886686 9541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "d156d815c0ca412bae6ea83aa1f1f927"
format_stamp: "Formatted at 2026-09-27 18:37:55 on dist-test-slave-bzzj"
server_key: "2b84b071b02cb61f732a85c994fa0f30"
server_key_iv: "9dbf22d565d788529cf3ed23d360646a"
server_key_version: "encryptionkey@0"
I20260927 18:37:55.886783 9541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:55.906098 9541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:55.908807 9541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:55.908954 9541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:55.909432 9541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:55.910144 9541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:55.910188 9541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.910235 9541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:55.910264 9541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:55.917863 9541 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37499
I20260927 18:37:55.917892 9669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37499 every 8 connection(s)
I20260927 18:37:55.918589 9541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:55.927474 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9541
I20260927 18:37:55.927568 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:55.927814 6160 external_mini_cluster.cc:1428] Setting key 01ae9a5b9a069c355900afe3bed0251a
I20260927 18:37:55.930418 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.920118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:58655 (local address 127.6.4.62:41277)
0927 18:37:55.920238 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:55.920241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:55.921005 (+ 764us) server_negotiation.cc:407] Connection header received
0927 18:37:55.922239 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:55.922242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:55.922295 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:55.922362 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:55.923367 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.923878 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.924957 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.925166 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.925259 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.927267 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:55.927272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:55.927274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:55.927301 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:55.928668 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.929089 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.929093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.929095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.929168 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.929458 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.929462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.929463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.929627 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:55.929707 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:55.930141 (+ 434us) server_negotiation.cc:299] Negotiation successful
0927 18:37:55.930272 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:37:55.931118 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.920441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:41277 (local address 127.6.4.3:58655)
0927 18:37:55.920858 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:55.920895 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:55.922038 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:55.922519 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:55.922525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:55.922817 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:55.923235 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:55.923248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.923983 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.923986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:55.924846 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:55.924855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.925244 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.925291 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:55.925442 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.925862 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:55.925876 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:55.927120 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:55.928766 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:55.928771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:55.928784 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:55.929003 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:55.929254 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:55.929256 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:55.929258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:55.929377 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:55.929712 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:55.929718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:55.929940 (+ 222us) client_negotiation.cc:769] Sending connection context
0927 18:37:55.930087 (+ 147us) client_negotiation.cc:240] Negotiation successful
0927 18:37:55.930395 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":100,"threads_started":1}
I20260927 18:37:55.932054 9670 heartbeater.cc:347] Connected to a master server at 127.6.4.62:41277
I20260927 18:37:55.932271 9670 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:55.932659 9670 heartbeater.cc:511] Master 127.6.4.62:41277 requested a full tablet report, sending...
I20260927 18:37:55.933485 9210 ts_manager.cc:194] Registered new tserver with Master: d156d815c0ca412bae6ea83aa1f1f927 (127.6.4.3:37499)
I20260927 18:37:55.934000 9210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:58655
I20260927 18:37:55.940886 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 18:37:55 dist-test-slave-bzzj krb5kdc[9166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534275, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:55.962744 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:55.955517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.6.4.62:41277)
0927 18:37:55.955636 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:55.955641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:55.955806 (+ 165us) server_negotiation.cc:407] Connection header received
0927 18:37:55.955918 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:55.955920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:55.955954 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:55.956010 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:55.956683 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.957163 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.957949 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:55.958153 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:55.958232 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:55.960203 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:55.960206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:55.960207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:55.960227 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:55.961313 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.961706 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.961708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.961710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.961776 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:55.962002 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:55.962005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:55.962007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:55.962193 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:55.962303 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:55.962506 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 18:37:55.962602 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260927 18:37:55.963035 9210 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42090
I20260927 18:37:55.963673 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9273
I20260927 18:37:55.968456 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9407
I20260927 18:37:55.972589 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9541
I20260927 18:37:55.976491 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9179
2026-09-27T18:37:55Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1947 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:56 dist-test-slave-bzzj krb5kdc[9682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:56 dist-test-slave-bzzj krb5kdc[9682](info): set up 2 sockets
Sep 27 18:37:56 dist-test-slave-bzzj krb5kdc[9682](info): commencing operation
krb5kdc: starting...
W20260927 18:37:56.915794 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:56 dist-test-slave-bzzj krb5kdc[9682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:57.026430 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40841
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:57.130328 9695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:57.130597 9695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:57.130650 9695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:57.134577 9695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:57.134629 9695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:57.134646 9695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:57.134661 9695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:57.134675 9695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:57.139699 9695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:57.140636 9695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:57.141607 9695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:57.149346 9700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.149426 9695 server_base.cc:1034] running on GCE node
W20260927 18:37:57.149363 9701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:57.149539 9703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.150128 9695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:57.151106 9695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:57.152252 9695 hybrid_clock.cc:648] HybridClock initialized: now 1790534277152236 us; error 30 us; skew 500 ppm
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:57.154778 9695 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:57.155565 9695 webserver.cc:460] Webserver started at http://127.6.4.62:33979/ using document root <none> and password file <none>
I20260927 18:37:57.156070 9695 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:57.156172 9695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:57.156370 9695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:57.158349 9695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "05bce0370ee348a7a5244bde440f27a7"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "fc62891b3e63646261d64defb38c9f1a"
server_key_iv: "95d93ca89da4175b0df52822112a7924"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.158869 9695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "05bce0370ee348a7a5244bde440f27a7"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "fc62891b3e63646261d64defb38c9f1a"
server_key_iv: "95d93ca89da4175b0df52822112a7924"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.162249 9695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:57.164410 9710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.165560 9695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:37:57.165664 9695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "05bce0370ee348a7a5244bde440f27a7"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "fc62891b3e63646261d64defb38c9f1a"
server_key_iv: "95d93ca89da4175b0df52822112a7924"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.165776 9695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:57.188683 9695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:57.191392 9695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:57.191557 9695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:57.197902 9695 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40841
I20260927 18:37:57.197947 9762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40841 every 8 connection(s)
I20260927 18:37:57.198773 9695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:57.201419 9763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:57.201777 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9695
I20260927 18:37:57.201900 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:57.202214 6160 external_mini_cluster.cc:1428] Setting key d648a33114494e484bfc67c599a6b530
I20260927 18:37:57.207991 9763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7: Bootstrap starting.
I20260927 18:37:57.210475 9763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7: Neither blocks nor log segments found. Creating new log.
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:57.211297 9763 log.cc:824] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:57.213754 9763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7: No bootstrap required, opened a new log
I20260927 18:37:57.216008 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.203459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.6.4.62:40841)
0927 18:37:57.203844 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:57.203858 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:57.203892 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:37:57.204564 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:57.204581 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:57.204897 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:57.205166 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:57.205960 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.206959 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.208063 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.208570 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.208680 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.211294 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:57.211300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:57.211336 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:57.211366 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:57.213346 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.213820 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.213826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.213831 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.213947 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.214196 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.214200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.214202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.214573 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:57.214742 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:57.215020 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 18:37:57.215326 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":263,"thread_start_us":111,"threads_started":1}
I20260927 18:37:57.217425 9763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bce0370ee348a7a5244bde440f27a7" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40841 } }
I20260927 18:37:57.217677 9763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:57.217731 9763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05bce0370ee348a7a5244bde440f27a7, State: Initialized, Role: FOLLOWER
I20260927 18:37:57.218133 9763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bce0370ee348a7a5244bde440f27a7" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40841 } }
I20260927 18:37:57.218254 9763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:57.218297 9763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:57.218364 9763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:57.219127 9763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bce0370ee348a7a5244bde440f27a7" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40841 } }
I20260927 18:37:57.219412 9763 leader_election.cc:302] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05bce0370ee348a7a5244bde440f27a7; no voters:
I20260927 18:37:57.219637 9763 leader_election.cc:288] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:37:57.219733 9768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:57.219918 9768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [term 1 LEADER]: Becoming Leader. State: Replica: 05bce0370ee348a7a5244bde440f27a7, State: Running, Role: LEADER
I20260927 18:37:57.220229 9768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bce0370ee348a7a5244bde440f27a7" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40841 } }
I20260927 18:37:57.220778 9763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:57.221841 9769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05bce0370ee348a7a5244bde440f27a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bce0370ee348a7a5244bde440f27a7" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40841 } } }
I20260927 18:37:57.221844 9770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05bce0370ee348a7a5244bde440f27a7. Latest consensus state: current_term: 1 leader_uuid: "05bce0370ee348a7a5244bde440f27a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05bce0370ee348a7a5244bde440f27a7" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40841 } } }
I20260927 18:37:57.221976 9769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:57.221993 9770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:57.222348 9777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:57.225090 9777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:57.230276 9777 catalog_manager.cc:1347] Generated new cluster ID: ca3262e7cfa54a13ab47e866ee254dec
I20260927 18:37:57.230337 9777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:57.246876 9777 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:57.247767 9777 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:57.261209 9777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05bce0370ee348a7a5244bde440f27a7: Generated new TSK 0
I20260927 18:37:57.261817 9777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:57.293184 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40841
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:57.395886 9789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:57.396126 9789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:57.396180 9789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:57.400020 9789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:57.400071 9789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:57.400166 9789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:57.405074 9789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:57.405944 9789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:57.406833 9789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:57.415753 9794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:57.415776 9797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.415864 9789 server_base.cc:1034] running on GCE node
W20260927 18:37:57.415776 9795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.416350 9789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:57.416903 9789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:57.418047 9789 hybrid_clock.cc:648] HybridClock initialized: now 1790534277418031 us; error 26 us; skew 500 ppm
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:57.420565 9789 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:57.421450 9789 webserver.cc:460] Webserver started at http://127.6.4.1:42195/ using document root <none> and password file <none>
I20260927 18:37:57.421988 9789 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:57.422057 9789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:57.422250 9789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:57.424176 9789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "fa612f074ca042e98984c4fc3a365a81"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "cdaa58a8b1f7a070b97357ccebe0e63a"
server_key_iv: "b26dd5a90f55aac9edba744fda76058a"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.424697 9789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "fa612f074ca042e98984c4fc3a365a81"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "cdaa58a8b1f7a070b97357ccebe0e63a"
server_key_iv: "b26dd5a90f55aac9edba744fda76058a"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.428038 9789 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:57.430153 9804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.431417 9789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 18:37:57.431509 9789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "fa612f074ca042e98984c4fc3a365a81"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "cdaa58a8b1f7a070b97357ccebe0e63a"
server_key_iv: "b26dd5a90f55aac9edba744fda76058a"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.431604 9789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:57.445050 9789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:57.447839 9789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:57.447992 9789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:57.448565 9789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:57.449306 9789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:57.449348 9789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.449396 9789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:57.449424 9789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.457420 9789 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:40909
I20260927 18:37:57.457446 9917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:40909 every 8 connection(s)
I20260927 18:37:57.458381 9789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:57.468621 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9789
I20260927 18:37:57.468727 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:57.468974 6160 external_mini_cluster.cc:1428] Setting key e78072829bdd8a5a93597de6c1cacc10
I20260927 18:37:57.471130 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.460056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:32801 (local address 127.6.4.62:40841)
0927 18:37:57.460192 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:57.460195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:57.460980 (+ 785us) server_negotiation.cc:407] Connection header received
0927 18:37:57.462314 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:57.462318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:57.462372 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:57.462441 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:57.463547 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.464056 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.465258 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.465479 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.465565 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.467708 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:57.467711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:57.467712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:57.467735 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:57.469177 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.469703 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.469707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.469709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.469785 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.470097 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.470101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.470102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.470282 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:57.470375 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:57.470867 (+ 492us) server_negotiation.cc:299] Negotiation successful
0927 18:37:57.470984 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:37:57.471916 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.460441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40841 (local address 127.6.4.1:32801)
0927 18:37:57.460846 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:57.460885 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:57.462109 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:57.462615 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:57.462621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:57.462959 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:57.463402 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:57.463414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.464182 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.464186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:57.465137 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:57.465147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.465561 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.465610 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:57.465769 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.466256 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:57.466271 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:57.467556 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:57.469296 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:57.469302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:57.469316 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:57.469626 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:57.469863 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:57.469866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:57.469868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:57.469993 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:57.470393 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:57.470399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:57.470646 (+ 247us) client_negotiation.cc:769] Sending connection context
0927 18:37:57.470794 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 18:37:57.471017 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":103,"threads_started":1}
I20260927 18:37:57.473021 9918 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40841
I20260927 18:37:57.473227 9918 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:57.473657 9918 heartbeater.cc:511] Master 127.6.4.62:40841 requested a full tablet report, sending...
I20260927 18:37:57.475052 9727 ts_manager.cc:194] Registered new tserver with Master: fa612f074ca042e98984c4fc3a365a81 (127.6.4.1:40909)
I20260927 18:37:57.476083 9727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:32801
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:57.492889 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40841
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:57.594910 9923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:57.595142 9923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:57.595201 9923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:57.599128 9923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:57.599187 9923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:57.599262 9923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:57.604199 9923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:57.605268 9923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:57.606226 9923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:57.615417 9931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:57.615509 9929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:57.615424 9928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.615783 9923 server_base.cc:1034] running on GCE node
I20260927 18:37:57.616205 9923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:57.616876 9923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:57.618048 9923 hybrid_clock.cc:648] HybridClock initialized: now 1790534277618025 us; error 66 us; skew 500 ppm
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:57.620847 9923 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:57.621731 9923 webserver.cc:460] Webserver started at http://127.6.4.2:38581/ using document root <none> and password file <none>
I20260927 18:37:57.622253 9923 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:57.622310 9923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:57.622488 9923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:57.624482 9923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "51cc31a23023400dbc994468cff7c523"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "3c44ddf7737b34176bf120bc434eff7c"
server_key_iv: "26d87e912a449eb7070ccbc571a91af9"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.624989 9923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "51cc31a23023400dbc994468cff7c523"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "3c44ddf7737b34176bf120bc434eff7c"
server_key_iv: "26d87e912a449eb7070ccbc571a91af9"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.628474 9923 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:37:57.630671 9938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.631763 9923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:57.631866 9923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "51cc31a23023400dbc994468cff7c523"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "3c44ddf7737b34176bf120bc434eff7c"
server_key_iv: "26d87e912a449eb7070ccbc571a91af9"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.631955 9923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:57.671602 9923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:57.674511 9923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:57.674675 9923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:57.675225 9923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:57.676008 9923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:57.676052 9923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.676126 9923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:57.676157 9923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.683944 9923 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:33337
I20260927 18:37:57.683971 10051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:33337 every 8 connection(s)
I20260927 18:37:57.684775 9923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:57.687847 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 9923
I20260927 18:37:57.687943 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:57.688225 6160 external_mini_cluster.cc:1428] Setting key 166ef7dd59511e3d41db0a966964d556
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
I20260927 18:37:57.697870 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.686382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:58423 (local address 127.6.4.62:40841)
0927 18:37:57.686513 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:57.686516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:57.687293 (+ 777us) server_negotiation.cc:407] Connection header received
0927 18:37:57.688658 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:57.688661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:57.688709 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:57.688797 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:57.690057 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.690770 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.691837 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.692203 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.692300 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.694621 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:57.694625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:57.694628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:57.694656 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:57.696081 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.696539 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.696543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.696545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.696633 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.696917 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.696921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.696923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.697128 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:57.697220 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:57.697654 (+ 434us) server_negotiation.cc:299] Negotiation successful
0927 18:37:57.697756 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:37:57.698627 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.686736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40841 (local address 127.6.4.2:58423)
0927 18:37:57.687157 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:57.687195 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:57.688456 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:57.688958 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:57.688967 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:57.689420 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:57.689924 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:57.689937 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.690864 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.690866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:57.691728 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:57.691736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.692287 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.692326 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:57.692462 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.692968 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:57.692983 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:57.694465 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:57.696205 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:57.696209 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:57.696220 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:57.696447 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:57.696715 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:57.696717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:57.696718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:57.696832 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:57.697210 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:57.697214 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:57.697433 (+ 219us) client_negotiation.cc:769] Sending connection context
0927 18:37:57.697582 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 18:37:57.697808 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:37:57.699740 10052 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40841
I20260927 18:37:57.699940 10052 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:57.700403 10052 heartbeater.cc:511] Master 127.6.4.62:40841 requested a full tablet report, sending...
I20260927 18:37:57.701311 9727 ts_manager.cc:194] Registered new tserver with Master: 51cc31a23023400dbc994468cff7c523 (127.6.4.2:33337)
I20260927 18:37:57.701836 9727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:58423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:57.712996 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40841
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:57.817669 10057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:57.817891 10057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:57.817930 10057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:57.821972 10057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:57.822026 10057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:57.822094 10057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:57.827201 10057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:57.828191 10057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:57.829136 10057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:57.838088 10062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.838151 10057 server_base.cc:1034] running on GCE node
W20260927 18:37:57.838095 10063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:57.838101 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:57.838639 10057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:57.839193 10057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:57.840337 10057 hybrid_clock.cc:648] HybridClock initialized: now 1790534277840324 us; error 23 us; skew 500 ppm
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:57.842733 10057 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:57.843503 10057 webserver.cc:460] Webserver started at http://127.6.4.3:35041/ using document root <none> and password file <none>
I20260927 18:37:57.843973 10057 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:57.844025 10057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:57.844228 10057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:57.846055 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "774a1bf5628c4af5978e8247fa5e1610"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "27d66df518e6d41123a1a37263dfce0d"
server_key_iv: "c11063d01141c5499e64dd1a6073ba05"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.846537 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "774a1bf5628c4af5978e8247fa5e1610"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "27d66df518e6d41123a1a37263dfce0d"
server_key_iv: "c11063d01141c5499e64dd1a6073ba05"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.849877 10057 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:57.852035 10072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.853174 10057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:57.853273 10057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "774a1bf5628c4af5978e8247fa5e1610"
format_stamp: "Formatted at 2026-09-27 18:37:57 on dist-test-slave-bzzj"
server_key: "27d66df518e6d41123a1a37263dfce0d"
server_key_iv: "c11063d01141c5499e64dd1a6073ba05"
server_key_version: "encryptionkey@0"
I20260927 18:37:57.853374 10057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:57.869536 10057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:57.872017 10057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:57.872197 10057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:57.872701 10057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:57.873461 10057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:57.873507 10057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.873543 10057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:57.873575 10057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:57.881373 10057 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:44113
I20260927 18:37:57.881397 10185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:44113 every 8 connection(s)
I20260927 18:37:57.882155 10057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:37:57.888298 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10057
I20260927 18:37:57.888382 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:57.888646 6160 external_mini_cluster.cc:1428] Setting key 0dfc47df32ccfe3b098b895849f5e427
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:57.894433 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.883710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:56247 (local address 127.6.4.62:40841)
0927 18:37:57.883821 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:57.883825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:57.884621 (+ 796us) server_negotiation.cc:407] Connection header received
0927 18:37:57.885883 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:57.885885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:57.885932 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:57.886004 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:57.887151 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.887653 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.888860 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.889109 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.889214 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.891778 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:57.891781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:57.891783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:57.891805 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:57.892945 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.893347 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.893349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.893350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.893415 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.893654 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.893656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.893657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.893805 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:57.893871 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:57.894236 (+ 365us) server_negotiation.cc:299] Negotiation successful
0927 18:37:57.894323 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:37:57.895187 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.884027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40841 (local address 127.6.4.3:56247)
0927 18:37:57.884478 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:57.884517 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:57.885695 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:57.886256 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:57.886263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:57.886589 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:57.887036 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:57.887048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.887753 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.887756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:57.888737 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:57.888746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.889206 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.889259 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:57.889434 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.890079 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:57.890100 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:57.891612 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:57.893037 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:57.893041 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:57.893051 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:57.893272 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:57.893472 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:57.893473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:57.893475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:57.893594 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:57.893869 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:57.893873 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:57.894067 (+ 194us) client_negotiation.cc:769] Sending connection context
0927 18:37:57.894207 (+ 140us) client_negotiation.cc:240] Negotiation successful
0927 18:37:57.894425 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":115,"threads_started":1}
I20260927 18:37:57.896296 10186 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40841
I20260927 18:37:57.896492 10186 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:57.896910 10186 heartbeater.cc:511] Master 127.6.4.62:40841 requested a full tablet report, sending...
I20260927 18:37:57.897795 9727 ts_manager.cc:194] Registered new tserver with Master: 774a1bf5628c4af5978e8247fa5e1610 (127.6.4.3:44113)
I20260927 18:37:57.898310 9727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:56247
I20260927 18:37:57.902323 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:37:57.925429 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.916962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.6.4.1:40909)
0927 18:37:57.917241 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:57.917249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:57.917274 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:37:57.917321 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:57.917325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:57.917497 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:57.917618 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:57.918227 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.919016 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.919860 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.920292 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.920386 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.922647 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:57.922655 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:57.922669 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:57.922693 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:57.923994 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.924386 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.924391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.924392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.924469 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.924657 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.924659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.924660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.924873 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:57.925025 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:57.925153 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:37:57.925271 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
Sep 27 18:37:57 dist-test-slave-bzzj krb5kdc[9682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:57.935243 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:57.928020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.6.4.62:40841)
0927 18:37:57.928170 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:57.928174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:57.928189 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:37:57.928301 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:57.928304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:57.928346 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:57.928407 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:57.928956 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.929457 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.930274 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:57.930483 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:57.930568 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:57.932644 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:57.932647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:57.932650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:57.932675 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:57.933742 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.934082 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.934086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.934087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.934167 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:57.934392 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:57.934396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:57.934397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:57.934646 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:57.934759 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:57.934982 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 18:37:57.935085 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260927 18:37:57.935627 9727 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55782
I20260927 18:37:57.936237 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9789
I20260927 18:37:57.940836 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9923
I20260927 18:37:57.945437 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10057
I20260927 18:37:57.949600 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 9695
2026-09-27T18:37:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1973 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:37:58 dist-test-slave-bzzj krb5kdc[10198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:37:58 dist-test-slave-bzzj krb5kdc[10198](info): set up 2 sockets
Sep 27 18:37:58 dist-test-slave-bzzj krb5kdc[10198](info): commencing operation
krb5kdc: starting...
W20260927 18:37:58.903689 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:37:58 dist-test-slave-bzzj krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:37:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:37:59.008767 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39189
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:39189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:59.110989 10211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:59.111212 10211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:59.111250 10211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:59.115198 10211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:37:59.115247 10211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:59.115264 10211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:59.115280 10211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:37:59.115293 10211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:37:59.120253 10211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:39189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:59.121120 10211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:59.122002 10211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:59.129491 10216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:59.129498 10219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:59.129575 10211 server_base.cc:1034] running on GCE node
W20260927 18:37:59.129505 10217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:59.130254 10211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:59.131201 10211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:59.132352 10211 hybrid_clock.cc:648] HybridClock initialized: now 1790534279132340 us; error 26 us; skew 500 ppm
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:59.135103 10211 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:37:59.135746 10211 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:59.135807 10211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:59.135978 10211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:59.137876 10211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "8a73f7fe371f46a6b404cf3f96c31617"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "7095d5fe828a444a1d0ad03e69f65152"
server_key_iv: "b51db174280f44ee0192a59c0eb85a01"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.138378 10211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "8a73f7fe371f46a6b404cf3f96c31617"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "7095d5fe828a444a1d0ad03e69f65152"
server_key_iv: "b51db174280f44ee0192a59c0eb85a01"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.141655 10211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260927 18:37:59.143767 10225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.144877 10211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:59.144987 10211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "8a73f7fe371f46a6b404cf3f96c31617"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "7095d5fe828a444a1d0ad03e69f65152"
server_key_iv: "b51db174280f44ee0192a59c0eb85a01"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.145071 10211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:59.163882 10211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:59.166383 10211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:59.166546 10211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:59.172462 10211 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:39189
I20260927 18:37:59.172717 10277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:39189 every 8 connection(s)
I20260927 18:37:59.173017 10211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:37:59.175483 10278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:59.180503 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: Bootstrap starting.
I20260927 18:37:59.182701 10278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:59.183269 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10211
I20260927 18:37:59.183380 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:37:59.183482 10278 log.cc:824] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:59.183661 6160 external_mini_cluster.cc:1428] Setting key 5abfffd4a8a06e603720fa1443dc7b78
I20260927 18:37:59.185477 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: No bootstrap required, opened a new log
I20260927 18:37:59.189587 10278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f7fe371f46a6b404cf3f96c31617" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39189 } }
I20260927 18:37:59.189826 10278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:59.189888 10278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a73f7fe371f46a6b404cf3f96c31617, State: Initialized, Role: FOLLOWER
I20260927 18:37:59.190339 10278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f7fe371f46a6b404cf3f96c31617" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39189 } }
I20260927 18:37:59.190486 10278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:37:59.190531 10278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:37:59.190593 10278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:37:59.191351 10278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f7fe371f46a6b404cf3f96c31617" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39189 } }
I20260927 18:37:59.191648 10278 leader_election.cc:302] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a73f7fe371f46a6b404cf3f96c31617; no voters:
I20260927 18:37:59.191905 10278 leader_election.cc:288] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [CANDIDATE]: Term 1 election: Requested vote from peers
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:59.191983 10283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:37:59.192163 10283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [term 1 LEADER]: Becoming Leader. State: Replica: 8a73f7fe371f46a6b404cf3f96c31617, State: Running, Role: LEADER
I20260927 18:37:59.192493 10283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f7fe371f46a6b404cf3f96c31617" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39189 } }
I20260927 18:37:59.193084 10278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:37:59.194046 10284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a73f7fe371f46a6b404cf3f96c31617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f7fe371f46a6b404cf3f96c31617" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39189 } } }
I20260927 18:37:59.194181 10284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:59.194526 10285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a73f7fe371f46a6b404cf3f96c31617. Latest consensus state: current_term: 1 leader_uuid: "8a73f7fe371f46a6b404cf3f96c31617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a73f7fe371f46a6b404cf3f96c31617" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39189 } } }
I20260927 18:37:59.194685 10285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617 [sys.catalog]: This master's current role is: LEADER
I20260927 18:37:59.197419 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.184638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.6.4.62:39189)
0927 18:37:59.185062 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.185082 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.185121 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:37:59.185890 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.185912 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.186289 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.186633 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:59.187178 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.188657 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.189457 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.190119 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.190275 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.192595 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:59.192602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:59.192617 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:59.192647 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:59.194662 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.195321 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.195327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.195333 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.195467 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.195710 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.195712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.195714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.196062 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:59.196260 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.196521 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.196782 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":116,"threads_started":1}
W20260927 18:37:59.197891 10298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:37:59.197966 10298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:37:59.198022 10292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:37:59.198827 10292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:37:59.203642 10292 catalog_manager.cc:1347] Generated new cluster ID: 141d8ac0e7224376aea4d93a8341bfa7
I20260927 18:37:59.203701 10292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:37:59.211903 10292 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:37:59.212698 10292 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:37:59.218762 10292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: Generated new TSK 0
I20260927 18:37:59.219412 10292 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:37:59.251103 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39189
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:59.390470 10304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:59.390699 10304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:59.390740 10304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:59.394789 10304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:59.394837 10304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:59.394905 10304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:37:59.399888 10304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:59.400796 10304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:59.401682 10304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:59.410121 10312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:59.410108 10310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:59.410239 10304 server_base.cc:1034] running on GCE node
W20260927 18:37:59.410360 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:59.410759 10304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:59.411309 10304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:59.412470 10304 hybrid_clock.cc:648] HybridClock initialized: now 1790534279412456 us; error 27 us; skew 500 ppm
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:59.415055 10304 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:37:59.415697 10304 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:59.415759 10304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:59.415927 10304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:59.417924 10304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "6adc3913b1fb4162a663ec5480ae3a06"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "a205f6bea20572359d07e77fc0316420"
server_key_iv: "47b161fd66d4ad47d7d834008e61ab4b"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.418478 10304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "6adc3913b1fb4162a663ec5480ae3a06"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "a205f6bea20572359d07e77fc0316420"
server_key_iv: "47b161fd66d4ad47d7d834008e61ab4b"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.421749 10304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:59.423748 10318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.424816 10304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:59.424907 10304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "6adc3913b1fb4162a663ec5480ae3a06"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "a205f6bea20572359d07e77fc0316420"
server_key_iv: "47b161fd66d4ad47d7d834008e61ab4b"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.424980 10304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:59.445628 10304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:59.448067 10304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:59.448212 10304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:59.448668 10304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:59.449359 10304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:59.449400 10304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.449435 10304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:59.449455 10304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.456933 10304 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:39889
I20260927 18:37:59.456954 10431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:39889 every 8 connection(s)
I20260927 18:37:59.457412 10304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:59.466492 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10304
I20260927 18:37:59.466603 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:37:59.466914 6160 external_mini_cluster.cc:1428] Setting key 882fdc94882f581fb72dcd55ea1b4e0a
I20260927 18:37:59.469304 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.458886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:35713 (local address 127.6.4.62:39189)
0927 18:37:59.459001 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.459004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.459709 (+ 705us) server_negotiation.cc:407] Connection header received
0927 18:37:59.460972 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.460975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.461031 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.461099 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:59.462125 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.462652 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.463706 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.463939 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.464023 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.466148 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:59.466152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:59.466154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:59.466177 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:59.467462 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.467898 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.467902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.467904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.467976 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.468327 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.468330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.468333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.468532 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:59.468624 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.469039 (+ 415us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.469152 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:37:59.469957 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.459177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39189 (local address 127.6.4.1:35713)
0927 18:37:59.459576 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:59.459613 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:59.460777 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:59.461257 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:59.461263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:59.461566 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:59.461988 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.462000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.462767 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.462769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.463589 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.463597 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.464019 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.464067 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.464236 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.464699 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:59.464713 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:59.465998 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:59.467574 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:59.467580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:59.467591 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:59.467808 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:59.468066 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:59.468069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:59.468071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:59.468244 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:59.468626 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:59.468632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:59.468828 (+ 196us) client_negotiation.cc:769] Sending connection context
0927 18:37:59.468966 (+ 138us) client_negotiation.cc:240] Negotiation successful
0927 18:37:59.469253 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":98,"threads_started":1}
I20260927 18:37:59.471089 10432 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39189
I20260927 18:37:59.471277 10432 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:59.471647 10432 heartbeater.cc:511] Master 127.6.4.62:39189 requested a full tablet report, sending...
I20260927 18:37:59.473093 10242 ts_manager.cc:194] Registered new tserver with Master: 6adc3913b1fb4162a663ec5480ae3a06 (127.6.4.1:39889)
I20260927 18:37:59.474637 10242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:35713
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:37:59.490396 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39189
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:59.591491 10437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:59.591706 10437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:59.591751 10437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:59.595710 10437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:59.595762 10437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:59.595817 10437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:37:59.600678 10437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:59.601527 10437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:59.602401 10437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:59.610777 10442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:59.610809 10445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:59.610859 10443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:59.611088 10437 server_base.cc:1034] running on GCE node
I20260927 18:37:59.611435 10437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:59.611969 10437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:59.613116 10437 hybrid_clock.cc:648] HybridClock initialized: now 1790534279613113 us; error 41 us; skew 500 ppm
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:59.615633 10437 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:37:59.616302 10437 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:59.616365 10437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:59.616528 10437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:59.618314 10437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "600bf0017c544cf196cc97cd5baa366c"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "90ace78b1434ece19e86256a5b7a7ede"
server_key_iv: "7747a0379d872f8c039709dbfe087602"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.618798 10437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "600bf0017c544cf196cc97cd5baa366c"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "90ace78b1434ece19e86256a5b7a7ede"
server_key_iv: "7747a0379d872f8c039709dbfe087602"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.622015 10437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:59.624073 10451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.625168 10437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:37:59.625262 10437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "600bf0017c544cf196cc97cd5baa366c"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "90ace78b1434ece19e86256a5b7a7ede"
server_key_iv: "7747a0379d872f8c039709dbfe087602"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.625344 10437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:59.648588 10437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:59.651261 10437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:59.651407 10437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:59.651889 10437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:59.652626 10437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:59.652685 10437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.652733 10437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:59.652756 10437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.660583 10437 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:37483
I20260927 18:37:59.660600 10564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:37483 every 8 connection(s)
I20260927 18:37:59.661068 10437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:37:59.665027 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10437
I20260927 18:37:59.665123 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:37:59.665385 6160 external_mini_cluster.cc:1428] Setting key ba86cda13e1ec6cbb4ac0f40715054f4
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:59.673343 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.662500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:42425 (local address 127.6.4.62:39189)
0927 18:37:59.662619 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.662622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.663356 (+ 734us) server_negotiation.cc:407] Connection header received
0927 18:37:59.664603 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.664606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.664649 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.664708 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:59.665889 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.666471 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.667549 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.667816 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.667914 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.670087 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:59.670091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:59.670093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:59.670120 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:59.671539 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.671960 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.671964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.671966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.672050 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.672367 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.672371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.672373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.672580 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:59.672675 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.673072 (+ 397us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.673189 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
I20260927 18:37:59.673983 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.662823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39189 (local address 127.6.4.2:42425)
0927 18:37:59.663222 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:59.663260 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:59.664417 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:59.664865 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:59.664871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:59.665226 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:59.665727 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.665748 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.666580 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.666583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.667438 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.667447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.667920 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.667970 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.668141 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.668610 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:59.668625 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:59.669928 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:59.671636 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:59.671640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:59.671650 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:59.671874 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:59.672171 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:59.672173 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:59.672174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:59.672287 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:59.672662 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:59.672666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:59.672863 (+ 197us) client_negotiation.cc:769] Sending connection context
0927 18:37:59.672996 (+ 133us) client_negotiation.cc:240] Negotiation successful
0927 18:37:59.673204 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
I20260927 18:37:59.674930 10565 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39189
I20260927 18:37:59.675118 10565 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:59.675561 10565 heartbeater.cc:511] Master 127.6.4.62:39189 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:37:59.676554 10242 ts_manager.cc:194] Registered new tserver with Master: 600bf0017c544cf196cc97cd5baa366c (127.6.4.2:37483)
I20260927 18:37:59.677176 10242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:42425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:37:59.689210 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39189
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:37:59.791285 10570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:37:59.791503 10570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:37:59.791543 10570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:37:59.795434 10570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:37:59.795488 10570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:37:59.795543 10570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:37:59.800434 10570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:37:59.801286 10570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:37:59.802157 10570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:37:59.810643 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:59.810631 10578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:37:59.810631 10575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:37:59.810925 10570 server_base.cc:1034] running on GCE node
I20260927 18:37:59.811276 10570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:37:59.811825 10570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:37:59.812968 10570 hybrid_clock.cc:648] HybridClock initialized: now 1790534279812952 us; error 23 us; skew 500 ppm
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:37:59.815487 10570 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:37:59.816214 10570 fs_manager.cc:359] Metadata directory not provided
I20260927 18:37:59.816282 10570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:37:59.816463 10570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:37:59.818324 10570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "77709cc25b7f4d4ca032090f5c2ab521"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "17597f95731d5fbca7349f3fd33d5c48"
server_key_iv: "48d6bbc9245556feb2ee3a2decf7dc2f"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.818837 10570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "77709cc25b7f4d4ca032090f5c2ab521"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "17597f95731d5fbca7349f3fd33d5c48"
server_key_iv: "48d6bbc9245556feb2ee3a2decf7dc2f"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.822161 10570 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:37:59.824286 10584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.825423 10570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:37:59.825531 10570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "77709cc25b7f4d4ca032090f5c2ab521"
format_stamp: "Formatted at 2026-09-27 18:37:59 on dist-test-slave-bzzj"
server_key: "17597f95731d5fbca7349f3fd33d5c48"
server_key_iv: "48d6bbc9245556feb2ee3a2decf7dc2f"
server_key_version: "encryptionkey@0"
I20260927 18:37:59.825659 10570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:37:59.840762 10570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:37:59.843240 10570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:37:59.843396 10570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:37:59.843912 10570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:37:59.844674 10570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:37:59.844718 10570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.844767 10570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:37:59.844796 10570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:37:59.852689 10570 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:45623
I20260927 18:37:59.852703 10697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:45623 every 8 connection(s)
I20260927 18:37:59.853175 10570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:37:59.853617 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10570
I20260927 18:37:59.853719 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:37:59.854036 6160 external_mini_cluster.cc:1428] Setting key 3d7355bf593775968d1eb515f9177662
Sep 27 18:37:59 dist-test-slave-bzzj krb5kdc[10198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534279, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:37:59.864854 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.854793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:59657 (local address 127.6.4.62:39189)
0927 18:37:59.854903 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.854906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.855887 (+ 981us) server_negotiation.cc:407] Connection header received
0927 18:37:59.857189 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.857192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.857234 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.857296 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:59.858337 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.858852 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.859860 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.860057 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.860166 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.862191 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:59.862194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:59.862196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:59.862217 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:59.863336 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.863725 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.863727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.863728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.863793 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.864029 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.864031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.864032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.864226 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:59.864298 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.864662 (+ 364us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.864746 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:37:59.865538 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.855258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39189 (local address 127.6.4.3:59657)
0927 18:37:59.855725 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:59.855777 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:59.857016 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:59.857431 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:59.857437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:59.857770 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:37:59.858214 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.858239 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.858936 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.858938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.859774 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.859782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.860152 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.860194 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.860337 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.860806 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:37:59.860822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:37:59.862055 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:37:59.863426 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:59.863430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:59.863439 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:59.863655 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:59.863853 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:37:59.863855 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:37:59.863856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:37:59.863964 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:37:59.864299 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:37:59.864303 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:37:59.864495 (+ 192us) client_negotiation.cc:769] Sending connection context
0927 18:37:59.864631 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 18:37:59.864846 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":105,"threads_started":1}
I20260927 18:37:59.866457 10698 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39189
I20260927 18:37:59.866680 10698 heartbeater.cc:464] Registering TS with master...
I20260927 18:37:59.867066 10698 heartbeater.cc:511] Master 127.6.4.62:39189 requested a full tablet report, sending...
I20260927 18:37:59.867901 10242 ts_manager.cc:194] Registered new tserver with Master: 77709cc25b7f4d4ca032090f5c2ab521 (127.6.4.3:45623)
I20260927 18:37:59.868423 10242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:59657
I20260927 18:37:59.878741 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:37:59.886649 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.880421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.6.4.62:39189)
0927 18:37:59.880528 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.880532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.880715 (+ 183us) server_negotiation.cc:407] Connection header received
0927 18:37:59.880832 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.880835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.880871 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.880929 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:37:59.881544 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.882020 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.882799 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.882996 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.883075 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.884236 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:37:59.884240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:37:59.884242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:37:59.884262 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:37:59.885251 (+ 989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.885607 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.885611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.885613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.885682 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:37:59.885905 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:37:59.885908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:37:59.885910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:37:59.886096 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:37:59.886211 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.886419 (+ 208us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.886515 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:37:59.888953 10242 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:37:59.891687 10242 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:37:59.904927 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.899473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:37483 (local address 127.0.0.1:34280)
0927 18:37:59.900569 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:59.900582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:59.900659 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:59.901311 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:59.901314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:59.901329 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:59.901404 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.901409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.902394 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.902397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.903403 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.903412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.904441 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.904445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.904545 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.904579 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 18:37:59.904682 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:37:59.904792 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":104,"threads_started":1}
I20260927 18:37:59.904960 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.899874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.6.4.2:37483)
0927 18:37:59.900540 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.900548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.900746 (+ 198us) server_negotiation.cc:407] Connection header received
0927 18:37:59.900818 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.900822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.901015 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.901156 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:59.901504 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.902266 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.903512 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.904324 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.904432 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.904566 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.904719 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.904814 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":83,"threads_started":1}
I20260927 18:37:59.905064 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.899074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:45623 (local address 127.0.0.1:54490)
0927 18:37:59.899392 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:59.899440 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:59.899549 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:59.900640 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:59.900646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:59.900671 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:59.900790 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.900796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.902151 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.902154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.903518 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.903528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.904625 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.904629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.904770 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.904792 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:37:59.904857 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:37:59.904941 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":22,"thread_start_us":76,"threads_started":1}
I20260927 18:37:59.905243 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.899186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39889 (local address 127.0.0.1:34954)
0927 18:37:59.900227 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0927 18:37:59.900240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:37:59.900327 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:37:59.901150 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:37:59.901153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:37:59.901169 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:37:59.901257 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.901263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.902579 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.902582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.903672 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:37:59.903679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.904643 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.904647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:37:59.904748 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.904768 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 18:37:59.904826 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:37:59.904880 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260927 18:37:59.905473 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.900229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.6.4.1:39889)
0927 18:37:59.900741 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.900754 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.900777 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:37:59.900847 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.900852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.901032 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.901159 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:59.901359 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.902353 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.903775 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.904349 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.904444 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.904575 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.904993 (+ 418us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.905327 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":104,"threads_started":1}
I20260927 18:37:59.905532 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:37:59.899713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.6.4.3:45623)
0927 18:37:59.900156 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0927 18:37:59.900172 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:37:59.900196 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:37:59.900266 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:37:59.900271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:37:59.900482 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:37:59.900640 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:37:59.900894 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.902018 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.903630 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:37:59.904385 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:37:59.904465 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:37:59.904566 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 18:37:59.905299 (+ 733us) server_negotiation.cc:299] Negotiation successful
0927 18:37:59.905392 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":128,"threads_started":1}
I20260927 18:37:59.907871 10366 tablet_service.cc:1467] Processing CreateTablet for tablet 021fea25588a4543b28ef1c2fb87d951 (DEFAULT_TABLE table=test-table [id=0e165e3f77e444e8b7cf0e4f56f14f64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:59.907897 10632 tablet_service.cc:1467] Processing CreateTablet for tablet 021fea25588a4543b28ef1c2fb87d951 (DEFAULT_TABLE table=test-table [id=0e165e3f77e444e8b7cf0e4f56f14f64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:59.907897 10499 tablet_service.cc:1467] Processing CreateTablet for tablet 021fea25588a4543b28ef1c2fb87d951 (DEFAULT_TABLE table=test-table [id=0e165e3f77e444e8b7cf0e4f56f14f64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:37:59.908784 10366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 021fea25588a4543b28ef1c2fb87d951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:59.908867 10499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 021fea25588a4543b28ef1c2fb87d951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:59.908875 10632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 021fea25588a4543b28ef1c2fb87d951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:37:59.914351 10715 tablet_bootstrap.cc:492] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: Bootstrap starting.
I20260927 18:37:59.916020 10717 tablet_bootstrap.cc:492] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Bootstrap starting.
I20260927 18:37:59.916424 10716 tablet_bootstrap.cc:492] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Bootstrap starting.
I20260927 18:37:59.917317 10715 tablet_bootstrap.cc:654] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:59.917693 10717 tablet_bootstrap.cc:654] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:59.918432 10716 tablet_bootstrap.cc:654] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Neither blocks nor log segments found. Creating new log.
I20260927 18:37:59.918480 10715 log.cc:824] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:59.918623 10717 log.cc:824] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:59.919622 10716 log.cc:824] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:37:59.920368 10717 tablet_bootstrap.cc:492] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: No bootstrap required, opened a new log
I20260927 18:37:59.920368 10715 tablet_bootstrap.cc:492] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: No bootstrap required, opened a new log
I20260927 18:37:59.920547 10717 ts_tablet_manager.cc:1397] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:37:59.920547 10715 ts_tablet_manager.cc:1397] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:37:59.921422 10716 tablet_bootstrap.cc:492] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: No bootstrap required, opened a new log
I20260927 18:37:59.921591 10716 ts_tablet_manager.cc:1397] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:37:59.924220 10715 raft_consensus.cc:348] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:37:59.924220 10717 raft_consensus.cc:348] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:37:59.924297 10716 raft_consensus.cc:348] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:37:59.924470 10715 raft_consensus.cc:374] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:59.924470 10717 raft_consensus.cc:374] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:59.924489 10716 raft_consensus.cc:374] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:37:59.924531 10716 raft_consensus.cc:729] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600bf0017c544cf196cc97cd5baa366c, State: Initialized, Role: FOLLOWER
I20260927 18:37:59.924525 10715 raft_consensus.cc:729] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6adc3913b1fb4162a663ec5480ae3a06, State: Initialized, Role: FOLLOWER
I20260927 18:37:59.924525 10717 raft_consensus.cc:729] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77709cc25b7f4d4ca032090f5c2ab521, State: Initialized, Role: FOLLOWER
I20260927 18:37:59.924937 10716 consensus_queue.cc:260] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:37:59.924966 10715 consensus_queue.cc:260] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:37:59.924966 10717 consensus_queue.cc:260] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:37:59.925516 10565 heartbeater.cc:503] Master 127.6.4.62:39189 was elected leader, sending a full tablet report...
I20260927 18:37:59.925824 10716 ts_tablet_manager.cc:1428] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:37:59.925824 10715 ts_tablet_manager.cc:1428] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:37:59.926146 10432 heartbeater.cc:503] Master 127.6.4.62:39189 was elected leader, sending a full tablet report...
I20260927 18:37:59.926425 10717 ts_tablet_manager.cc:1428] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:37:59.926460 10698 heartbeater.cc:503] Master 127.6.4.62:39189 was elected leader, sending a full tablet report...
W20260927 18:37:59.958361 10433 tablet.cc:2369] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:00.104046 10699 tablet.cc:2369] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:00.162024 10566 tablet.cc:2369] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:00.285050 10721 raft_consensus.cc:482] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:00.285269 10721 raft_consensus.cc:504] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:38:00.286446 10721 leader_election.cc:288] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77709cc25b7f4d4ca032090f5c2ab521 (127.6.4.3:45623), 6adc3913b1fb4162a663ec5480ae3a06 (127.6.4.1:39889)
I20260927 18:38:00.290124 10714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.286814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:44067 (local address 127.6.4.1:39889)
0927 18:38:00.286997 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:00.287002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:00.287047 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:38:00.287231 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:00.287234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:00.287278 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:00.287339 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:00.287603 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.288151 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.289261 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.289636 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.289733 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:00.289793 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:00.289942 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:38:00.290004 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260927 18:38:00.290143 10712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.287070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:38179 (local address 127.6.4.3:45623)
0927 18:38:00.287199 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:00.287202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:00.287212 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:00.287261 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:00.287263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:00.287299 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:00.287360 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:00.287634 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.288137 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.289266 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.289651 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.289739 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:00.289796 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:00.289964 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 18:38:00.290024 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 18:38:00.290163 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.286663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:45623 (local address 127.6.4.2:38179)
0927 18:38:00.286941 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:00.286956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:00.287060 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:00.287386 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:00.287389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:00.287425 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:00.287531 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:00.287536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.288239 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.288242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:00.289150 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:00.289159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.289762 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.289766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:00.289862 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:00.289878 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:38:00.289942 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:38:00.290007 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":70,"spinlock_wait_cycles":768,"thread_start_us":123,"threads_started":1}
I20260927 18:38:00.290174 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.286708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39889 (local address 127.6.4.2:44067)
0927 18:38:00.286966 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:00.286980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:00.287080 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:00.287362 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:00.287365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:00.287380 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:00.287502 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:00.287507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.288250 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.288253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:00.289147 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:00.289155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.289739 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.289743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:00.289832 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:00.289855 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:38:00.289916 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:38:00.289984 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":24,"spinlock_wait_cycles":3968,"thread_start_us":65,"threads_started":1}
I20260927 18:38:00.290967 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021fea25588a4543b28ef1c2fb87d951" candidate_uuid: "600bf0017c544cf196cc97cd5baa366c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77709cc25b7f4d4ca032090f5c2ab521" is_pre_election: true
I20260927 18:38:00.290966 10386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021fea25588a4543b28ef1c2fb87d951" candidate_uuid: "600bf0017c544cf196cc97cd5baa366c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6adc3913b1fb4162a663ec5480ae3a06" is_pre_election: true
I20260927 18:38:00.291224 10652 raft_consensus.cc:2454] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600bf0017c544cf196cc97cd5baa366c in term 0.
I20260927 18:38:00.291231 10386 raft_consensus.cc:2454] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600bf0017c544cf196cc97cd5baa366c in term 0.
I20260927 18:38:00.291623 10453 leader_election.cc:302] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 600bf0017c544cf196cc97cd5baa366c, 77709cc25b7f4d4ca032090f5c2ab521; no voters:
I20260927 18:38:00.291853 10721 raft_consensus.cc:2781] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:00.291929 10721 raft_consensus.cc:482] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:00.291975 10721 raft_consensus.cc:3037] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:00.292946 10721 raft_consensus.cc:504] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:38:00.293243 10721 leader_election.cc:288] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [CANDIDATE]: Term 1 election: Requested vote from peers 77709cc25b7f4d4ca032090f5c2ab521 (127.6.4.3:45623), 6adc3913b1fb4162a663ec5480ae3a06 (127.6.4.1:39889)
I20260927 18:38:00.293545 10386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021fea25588a4543b28ef1c2fb87d951" candidate_uuid: "600bf0017c544cf196cc97cd5baa366c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6adc3913b1fb4162a663ec5480ae3a06"
I20260927 18:38:00.293617 10652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021fea25588a4543b28ef1c2fb87d951" candidate_uuid: "600bf0017c544cf196cc97cd5baa366c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77709cc25b7f4d4ca032090f5c2ab521"
I20260927 18:38:00.293666 10386 raft_consensus.cc:3037] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:00.293727 10652 raft_consensus.cc:3037] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:00.294615 10386 raft_consensus.cc:2454] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600bf0017c544cf196cc97cd5baa366c in term 1.
I20260927 18:38:00.294808 10652 raft_consensus.cc:2454] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600bf0017c544cf196cc97cd5baa366c in term 1.
I20260927 18:38:00.294878 10454 leader_election.cc:302] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 600bf0017c544cf196cc97cd5baa366c, 6adc3913b1fb4162a663ec5480ae3a06; no voters:
I20260927 18:38:00.295023 10721 raft_consensus.cc:2781] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:00.295262 10721 raft_consensus.cc:686] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 1 LEADER]: Becoming Leader. State: Replica: 600bf0017c544cf196cc97cd5baa366c, State: Running, Role: LEADER
I20260927 18:38:00.295550 10721 consensus_queue.cc:237] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } }
I20260927 18:38:00.298378 10241 catalog_manager.cc:5495] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c reported cstate change: term changed from 0 to 1, leader changed from <none> to 600bf0017c544cf196cc97cd5baa366c (127.6.4.2). New cstate: current_term: 1 leader_uuid: "600bf0017c544cf196cc97cd5baa366c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "600bf0017c544cf196cc97cd5baa366c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 37483 } health_report { overall_health: HEALTHY } } }
I20260927 18:38:00.350874 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.347757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.6.4.2:37483)
0927 18:38:00.347872 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:00.347875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:00.347889 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:38:00.348033 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:00.348036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:00.348110 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:00.348197 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:00.348379 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.348907 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.349662 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:00.349881 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:00.349953 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:00.350160 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:00.350564 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:00.350647 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:00.350705 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 18:38:00.350771 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:38:00.356307 10652 raft_consensus.cc:1261] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 1 FOLLOWER]: Refusing update from remote peer 600bf0017c544cf196cc97cd5baa366c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:00.356333 10386 raft_consensus.cc:1261] T 021fea25588a4543b28ef1c2fb87d951 P 6adc3913b1fb4162a663ec5480ae3a06 [term 1 FOLLOWER]: Refusing update from remote peer 600bf0017c544cf196cc97cd5baa366c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:00.356930 10721 consensus_queue.cc:1035] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [LEADER]: Connected to new peer: Peer: permanent_uuid: "77709cc25b7f4d4ca032090f5c2ab521" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:00.357105 10726 consensus_queue.cc:1035] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6adc3913b1fb4162a663ec5480ae3a06" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:00.372720 10241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:00.372901 10241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:00.375638 10241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a73f7fe371f46a6b404cf3f96c31617: Sending DeleteTablet for 3 replicas of tablet 021fea25588a4543b28ef1c2fb87d951
I20260927 18:38:00.376341 10366 tablet_service.cc:1514] Processing DeleteTablet for tablet 021fea25588a4543b28ef1c2fb87d951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:00 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:34954
I20260927 18:38:00.376364 10632 tablet_service.cc:1514] Processing DeleteTablet for tablet 021fea25588a4543b28ef1c2fb87d951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:00 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:54490
I20260927 18:38:00.376400 10499 tablet_service.cc:1514] Processing DeleteTablet for tablet 021fea25588a4543b28ef1c2fb87d951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:00 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:34280
I20260927 18:38:00.376952 10739 tablet_replica.cc:330] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: stopping tablet replica
I20260927 18:38:00.377218 10739 raft_consensus.cc:2229] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:00.377346 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10304
I20260927 18:38:00.377862 10739 raft_consensus.cc:2258] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:00.378686 10741 tablet_replica.cc:330] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: stopping tablet replica
I20260927 18:38:00.379583 10739 ts_tablet_manager.cc:1905] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:00.382052 10739 ts_tablet_manager.cc:1918] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:00.382139 10739 log.cc:1196] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/021fea25588a4543b28ef1c2fb87d951
I20260927 18:38:00.382409 10739 ts_tablet_manager.cc:1939] T 021fea25588a4543b28ef1c2fb87d951 P 77709cc25b7f4d4ca032090f5c2ab521: Deleting consensus metadata
I20260927 18:38:00.383416 10227 catalog_manager.cc:4841] TS 77709cc25b7f4d4ca032090f5c2ab521 (127.6.4.3:45623): tablet 021fea25588a4543b28ef1c2fb87d951 (table test-table [id=0e165e3f77e444e8b7cf0e4f56f14f64]) successfully deleted
I20260927 18:38:00.383785 10741 raft_consensus.cc:2229] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:00.384142 10741 raft_consensus.cc:2258] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 18:38:00.385301 10228 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:39889 (error 108)
I20260927 18:38:00.385327 10741 ts_tablet_manager.cc:1905] T 021fea25588a4543b28ef1c2fb87d951 P 600bf0017c544cf196cc97cd5baa366c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:00.386127 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.385895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39889 (local address 127.0.0.1:34960)
0927 18:38:00.385998 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:00.386056 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:39889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:38:00.386240 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10437
W20260927 18:38:00.389024 10228 catalog_manager.cc:4584] TS 6adc3913b1fb4162a663ec5480ae3a06 (127.6.4.1:39889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 021fea25588a4543b28ef1c2fb87d951: Network error: Client connection negotiation failed: client connection to 127.6.4.1:39889: connect: Connection refused (error 111)
I20260927 18:38:00.392408 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10570
I20260927 18:38:00.393177 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.392971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:37483 (local address 127.0.0.1:34298)
0927 18:38:00.393072 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:00.393119 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.2:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 18:38:00.397505 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10211
2026-09-27T18:38:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2448 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T18:38:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:00Z Disabled control of system clock
I20260927 18:38:00.423689 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39963
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:38:00.524696 10746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:00.524920 10746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:00.524953 10746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:00.528746 10746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:00.528789 10746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:00.528803 10746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:00.528817 10746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:00.528831 10746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:00.533268 10746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:39963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:00.534121 10746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:00.535270 10746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:00.542887 10754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:00.542886 10751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:00.542896 10752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:00.543136 10746 server_base.cc:1034] running on GCE node
I20260927 18:38:00.543612 10746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:00.544588 10746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:00.545750 10746 hybrid_clock.cc:648] HybridClock initialized: now 1790534280545727 us; error 36 us; skew 500 ppm
I20260927 18:38:00.547382 10746 webserver.cc:460] Webserver started at http://127.6.4.62:40879/ using document root <none> and password file <none>
I20260927 18:38:00.547902 10746 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:00.547964 10746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:00.548168 10746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:00.550001 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "471c47dcbab64ae39cf2cfef3ef464ef"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "178920443e5b9cc9aeb67dc20e8cb012"
server_key_iv: "31ebb72489358a0ab4f11dc63776dace"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.550490 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "471c47dcbab64ae39cf2cfef3ef464ef"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "178920443e5b9cc9aeb67dc20e8cb012"
server_key_iv: "31ebb72489358a0ab4f11dc63776dace"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.553777 10746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:00.555902 10760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.557017 10746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:00.557119 10746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "471c47dcbab64ae39cf2cfef3ef464ef"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "178920443e5b9cc9aeb67dc20e8cb012"
server_key_iv: "31ebb72489358a0ab4f11dc63776dace"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.557204 10746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:00.561390 10746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:00.561905 10746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:00.562043 10746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:00.568048 10746 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:39963
I20260927 18:38:00.568053 10812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:39963 every 8 connection(s)
I20260927 18:38:00.568849 10746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:00.571583 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:00.576460 10813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef: Bootstrap starting.
I20260927 18:38:00.578114 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10746
I20260927 18:38:00.578219 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:00.578454 6160 external_mini_cluster.cc:1428] Setting key 3da30a6e1471b6e3849c57e824a69a38
I20260927 18:38:00.578670 10813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:00.579432 10813 log.cc:824] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:00.581173 10813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef: No bootstrap required, opened a new log
I20260927 18:38:00.583089 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.579394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.6.4.62:39963)
0927 18:38:00.580018 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:00.580036 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:00.580077 (+ 41us) server_negotiation.cc:407] Connection header received
0927 18:38:00.580698 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:00.580712 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:00.580984 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:00.581264 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:00.581979 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:00.581986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:00.581996 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:00.582006 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:00.582055 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:00.582070 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:00.582156 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:00.582261 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 18:38:00.582484 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":106,"threads_started":1}
I20260927 18:38:00.584709 10813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39963 } }
I20260927 18:38:00.584937 10813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:00.584991 10813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471c47dcbab64ae39cf2cfef3ef464ef, State: Initialized, Role: FOLLOWER
I20260927 18:38:00.585348 10813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39963 } }
I20260927 18:38:00.585465 10813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:00.585523 10813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:00.585614 10813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:00.586333 10813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39963 } }
I20260927 18:38:00.586611 10813 leader_election.cc:302] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 471c47dcbab64ae39cf2cfef3ef464ef; no voters:
I20260927 18:38:00.586848 10813 leader_election.cc:288] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:00.586928 10818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:00.587121 10818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [term 1 LEADER]: Becoming Leader. State: Replica: 471c47dcbab64ae39cf2cfef3ef464ef, State: Running, Role: LEADER
I20260927 18:38:00.587424 10818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39963 } }
I20260927 18:38:00.587877 10813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:00.588850 10819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39963 } } }
I20260927 18:38:00.588958 10819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:00.589214 10820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 471c47dcbab64ae39cf2cfef3ef464ef. Latest consensus state: current_term: 1 leader_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471c47dcbab64ae39cf2cfef3ef464ef" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39963 } } }
I20260927 18:38:00.589290 10827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:00.591921 10827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:00.589310 10820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:00.596513 10827 catalog_manager.cc:1347] Generated new cluster ID: 47c4f89d96a54ec7aefad33874ca8efd
I20260927 18:38:00.596566 10827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:00.610006 10827 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:00.610272 10827 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:00.617626 10827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef: Generated new TSK 0
I20260927 18:38:00.618273 10827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:38:00.623878 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39963
--builtin_ntp_servers=127.6.4.20:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:38:00.724649 10837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:00.724855 10837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:00.724890 10837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:00.728695 10837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:00.728734 10837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:00.728783 10837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:00.733189 10837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:00.734019 10837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:00.735153 10837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:00.743449 10842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:00.743543 10845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:00.743458 10843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:00.743606 10837 server_base.cc:1034] running on GCE node
I20260927 18:38:00.744009 10837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:00.744572 10837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:00.745714 10837 hybrid_clock.cc:648] HybridClock initialized: now 1790534280745697 us; error 27 us; skew 500 ppm
I20260927 18:38:00.747275 10837 webserver.cc:460] Webserver started at http://127.6.4.1:39737/ using document root <none> and password file <none>
I20260927 18:38:00.747746 10837 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:00.747799 10837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:00.747969 10837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:00.749787 10837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "2b2ec4c64f2b4505975c920599a52330"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "f0fe152afa51fdd54d5fd79711e2f26b"
server_key_iv: "f9bc9c35cfcb406226f1b9744ae61a87"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.750276 10837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "2b2ec4c64f2b4505975c920599a52330"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "f0fe152afa51fdd54d5fd79711e2f26b"
server_key_iv: "f9bc9c35cfcb406226f1b9744ae61a87"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.753434 10837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260927 18:38:00.755573 10851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.756645 10837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260927 18:38:00.756739 10837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "2b2ec4c64f2b4505975c920599a52330"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "f0fe152afa51fdd54d5fd79711e2f26b"
server_key_iv: "f9bc9c35cfcb406226f1b9744ae61a87"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.756826 10837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:00.760902 10837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:00.761400 10837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:00.761536 10837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:00.761986 10837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:00.762743 10837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:00.762786 10837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.762832 10837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:00.762861 10837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.770424 10837 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:42007
I20260927 18:38:00.770447 10964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:42007 every 8 connection(s)
I20260927 18:38:00.771164 10837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:00.776355 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.772760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:50395 (local address 127.6.4.62:39963)
0927 18:38:00.772880 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:00.772884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:00.773844 (+ 960us) server_negotiation.cc:407] Connection header received
0927 18:38:00.774986 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:00.774989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:00.775023 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:00.775089 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:00.775981 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:00.775984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:00.775986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:00.775996 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:00.776015 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:00.776022 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:00.776111 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:00.776219 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 18:38:00.776253 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:38:00.777086 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.773096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39963 (local address 127.6.4.1:50395)
0927 18:38:00.773719 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:00.773756 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:00.774821 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:00.775224 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:00.775230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:00.775311 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:00.775758 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:00.775759 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:00.775793 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:00.775854 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:00.776082 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:00.776086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:00.776108 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:38:00.776240 (+ 132us) client_negotiation.cc:240] Negotiation successful
0927 18:38:00.776411 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":103,"threads_started":1}
I20260927 18:38:00.778071 10965 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39963
I20260927 18:38:00.778257 10965 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:00.778591 10965 heartbeater.cc:511] Master 127.6.4.62:39963 requested a full tablet report, sending...
I20260927 18:38:00.778820 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10837
I20260927 18:38:00.778923 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:00.779206 6160 external_mini_cluster.cc:1428] Setting key dad43f00d07bd7ff6775fdbd3bc8d841
I20260927 18:38:00.780022 10777 ts_manager.cc:194] Registered new tserver with Master: 2b2ec4c64f2b4505975c920599a52330 (127.6.4.1:42007)
I20260927 18:38:00.781145 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39963
--builtin_ntp_servers=127.6.4.20:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:38:00.881484 10968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:00.881695 10968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:00.881738 10968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:00.885567 10968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:00.885607 10968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:00.885665 10968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:00.890177 10968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:00.891017 10968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:00.892200 10968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:00.900553 10974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:00.900527 10973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:00.900537 10976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:00.900697 10968 server_base.cc:1034] running on GCE node
I20260927 18:38:00.901104 10968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:00.901619 10968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:00.902767 10968 hybrid_clock.cc:648] HybridClock initialized: now 1790534280902750 us; error 28 us; skew 500 ppm
I20260927 18:38:00.908036 10968 webserver.cc:460] Webserver started at http://127.6.4.2:43197/ using document root <none> and password file <none>
I20260927 18:38:00.908548 10968 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:00.908609 10968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:00.908790 10968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:00.910574 10968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "80b44e999ecd4aacb64bcfbe5e178d87"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "a2e2a084a14955126f6104bfa216ae03"
server_key_iv: "d49d26105a4abd02509e5a75d719696e"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.911068 10968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "80b44e999ecd4aacb64bcfbe5e178d87"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "a2e2a084a14955126f6104bfa216ae03"
server_key_iv: "d49d26105a4abd02509e5a75d719696e"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.914255 10968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:00.916306 10982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.917389 10968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:00.917490 10968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "80b44e999ecd4aacb64bcfbe5e178d87"
format_stamp: "Formatted at 2026-09-27 18:38:00 on dist-test-slave-bzzj"
server_key: "a2e2a084a14955126f6104bfa216ae03"
server_key_iv: "d49d26105a4abd02509e5a75d719696e"
server_key_version: "encryptionkey@0"
I20260927 18:38:00.917570 10968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:00.921561 10968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:00.922056 10968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:00.922192 10968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:00.922636 10968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:00.923394 10968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:00.923439 10968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.923483 10968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:00.923509 10968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:00.930987 10968 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:34311
I20260927 18:38:00.931010 11095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:34311 every 8 connection(s)
I20260927 18:38:00.931702 10968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:00.935936 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 10968
I20260927 18:38:00.936054 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:00.936370 6160 external_mini_cluster.cc:1428] Setting key 88c88aae8b637f38454b2e95883c8429
I20260927 18:38:00.937095 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.933219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51649 (local address 127.6.4.62:39963)
0927 18:38:00.933368 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:00.933373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:00.934294 (+ 921us) server_negotiation.cc:407] Connection header received
0927 18:38:00.935416 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:00.935419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:00.935454 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:00.935519 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:00.936665 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:00.936667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:00.936669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:00.936681 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:00.936699 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:00.936707 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:00.936798 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:00.936933 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 18:38:00.936972 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 18:38:00.937825 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:00.933566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39963 (local address 127.6.4.2:51649)
0927 18:38:00.934167 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:00.934203 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:00.935260 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:00.935656 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:00.935663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:00.935791 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:00.936347 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:00.936349 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:00.936397 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:00.936478 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:00.936800 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:00.936809 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:00.936816 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:38:00.936961 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 18:38:00.937169 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
I20260927 18:38:00.938143 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39963
--builtin_ntp_servers=127.6.4.20:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 18:38:00.938764 11096 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39963
I20260927 18:38:00.938984 11096 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:00.939392 11096 heartbeater.cc:511] Master 127.6.4.62:39963 requested a full tablet report, sending...
I20260927 18:38:00.940436 10777 ts_manager.cc:194] Registered new tserver with Master: 80b44e999ecd4aacb64bcfbe5e178d87 (127.6.4.2:34311)
W20260927 18:38:01.039691 11099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:01.039917 11099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:01.039954 11099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:01.043772 11099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:01.043816 11099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:01.043867 11099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:01.048353 11099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:01.049196 11099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:01.050382 11099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:01.059062 11107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:01.059056 11104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:01.059085 11105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:01.059180 11099 server_base.cc:1034] running on GCE node
I20260927 18:38:01.059679 11099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:01.060324 11099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:01.061475 11099 hybrid_clock.cc:648] HybridClock initialized: now 1790534281061434 us; error 52 us; skew 500 ppm
I20260927 18:38:01.063114 11099 webserver.cc:460] Webserver started at http://127.6.4.3:38363/ using document root <none> and password file <none>
I20260927 18:38:01.063627 11099 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:01.063688 11099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:01.063872 11099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:01.065786 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "5ff2748df22f45e3b53de26615e62aa2"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "29b412f828a34ba5255835ee3dc6da04"
server_key_iv: "cc6354ec8b7d8687db2d2827d8b61b89"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.066282 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "5ff2748df22f45e3b53de26615e62aa2"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "29b412f828a34ba5255835ee3dc6da04"
server_key_iv: "cc6354ec8b7d8687db2d2827d8b61b89"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.069538 11099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260927 18:38:01.071575 11113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:01.072687 11099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:01.072791 11099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "5ff2748df22f45e3b53de26615e62aa2"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "29b412f828a34ba5255835ee3dc6da04"
server_key_iv: "cc6354ec8b7d8687db2d2827d8b61b89"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.072873 11099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:01.076970 11099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:01.077474 11099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:01.077616 11099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:01.078092 11099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:01.078876 11099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:01.078918 11099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:01.078965 11099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:01.078994 11099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:01.086628 11099 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37467
I20260927 18:38:01.086655 11226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37467 every 8 connection(s)
I20260927 18:38:01.087368 11099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:01.092870 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.089020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:33673 (local address 127.6.4.62:39963)
0927 18:38:01.089132 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.089135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.090164 (+ 1029us) server_negotiation.cc:407] Connection header received
0927 18:38:01.091319 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.091321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.091356 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.091422 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.092401 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.092404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.092407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.092421 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.092442 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.092450 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.092526 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.092697 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.092742 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:38:01.093542 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11099
I20260927 18:38:01.093659 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:01.093762 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.089362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39963 (local address 127.6.4.3:33673)
0927 18:38:01.090025 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.090061 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.091140 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.091541 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.091546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.091643 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:01.092115 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:01.092117 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:01.092168 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:01.092252 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:01.092519 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:01.092526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:01.092532 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:01.092724 (+ 192us) client_negotiation.cc:240] Negotiation successful
0927 18:38:01.092946 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":113,"threads_started":1}
I20260927 18:38:01.093957 6160 external_mini_cluster.cc:1428] Setting key 039e38d20289618f0f721fc417ecf02e
I20260927 18:38:01.094831 11227 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39963
I20260927 18:38:01.095062 11227 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:01.095425 11227 heartbeater.cc:511] Master 127.6.4.62:39963 requested a full tablet report, sending...
I20260927 18:38:01.096510 10777 ts_manager.cc:194] Registered new tserver with Master: 5ff2748df22f45e3b53de26615e62aa2 (127.6.4.3:37467)
I20260927 18:38:01.107177 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:38:01.110050 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.108803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.6.4.62:39963)
0927 18:38:01.108927 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.108931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.109052 (+ 121us) server_negotiation.cc:407] Connection header received
0927 18:38:01.109143 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.109145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.109173 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.109239 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.109774 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.109776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.109778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.109787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.109805 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.109812 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.109871 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.109916 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.109945 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:38:01.112353 10777 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:01.115058 10777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:01.125474 11235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.122090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37467 (local address 127.0.0.1:35924)
0927 18:38:01.122716 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.122762 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.122851 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.124167 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.124170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.124184 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:01.124513 (+ 329us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:01.124515 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:01.124524 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:01.124547 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:01.125246 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:01.125249 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:01.125252 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:01.125297 (+ 45us) client_negotiation.cc:240] Negotiation successful
0927 18:38:01.125325 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":58,"threads_started":1}
I20260927 18:38:01.125376 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.122194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:42007 (local address 127.0.0.1:58510)
0927 18:38:01.122840 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.122852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.122926 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.123706 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.123712 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.123784 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:01.124289 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:01.124291 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:01.124310 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:01.124374 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:01.125077 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:01.125085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:01.125090 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:01.125194 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 18:38:01.125258 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":62,"threads_started":1}
I20260927 18:38:01.125619 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.123015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58510 (local address 127.6.4.1:42007)
0927 18:38:01.123276 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.123286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.123310 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:38:01.123361 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.123366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.123555 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.124033 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.124912 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.124918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.124929 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.124938 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.124975 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.124997 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.125111 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.125395 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.125462 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
I20260927 18:38:01.125509 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.122842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.6.4.2:34311)
0927 18:38:01.123126 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.123135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.123532 (+ 397us) server_negotiation.cc:407] Connection header received
0927 18:38:01.123591 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.123597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.123779 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.123904 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.125017 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.125024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.125035 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.125045 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.125086 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.125117 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.125216 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.125300 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.125364 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":107,"threads_started":1}
I20260927 18:38:01.125741 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.122672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:34311 (local address 127.0.0.1:48794)
0927 18:38:01.123432 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.123444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.123517 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.124535 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.124537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.124549 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:01.124871 (+ 322us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:01.124873 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:01.124883 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:01.124909 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:01.125206 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:01.125209 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:01.125212 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:01.125289 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:38:01.125333 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":2,"thread_start_us":42,"threads_started":1}
I20260927 18:38:01.126490 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.122148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.6.4.3:37467)
0927 18:38:01.123762 (+ 1614us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.123772 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.123791 (+ 19us) server_negotiation.cc:407] Connection header received
0927 18:38:01.123833 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.123838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.123997 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.124369 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.125066 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.125071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.125080 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.125090 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.125132 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.125156 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.125445 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.125481 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.126327 (+ 846us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1525,"thread_start_us":73,"threads_started":1}
I20260927 18:38:01.128171 10899 tablet_service.cc:1467] Processing CreateTablet for tablet e52ffe7848a74d00a94076670df829d7 (DEFAULT_TABLE table=test-table [id=05804aab655944c587b98689c73d0ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:01.128281 11030 tablet_service.cc:1467] Processing CreateTablet for tablet e52ffe7848a74d00a94076670df829d7 (DEFAULT_TABLE table=test-table [id=05804aab655944c587b98689c73d0ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:01.129029 10899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e52ffe7848a74d00a94076670df829d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:01.128870 11161 tablet_service.cc:1467] Processing CreateTablet for tablet e52ffe7848a74d00a94076670df829d7 (DEFAULT_TABLE table=test-table [id=05804aab655944c587b98689c73d0ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:01.129212 11030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e52ffe7848a74d00a94076670df829d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:01.129714 11161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e52ffe7848a74d00a94076670df829d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:01.134511 11241 tablet_bootstrap.cc:492] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: Bootstrap starting.
I20260927 18:38:01.136417 11243 tablet_bootstrap.cc:492] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Bootstrap starting.
I20260927 18:38:01.136685 11241 tablet_bootstrap.cc:654] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:01.137332 11241 log.cc:824] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:01.138010 11243 tablet_bootstrap.cc:654] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:01.138603 11243 log.cc:824] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:01.138886 11241 tablet_bootstrap.cc:492] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: No bootstrap required, opened a new log
I20260927 18:38:01.139077 11241 ts_tablet_manager.cc:1397] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:38:01.139111 11242 tablet_bootstrap.cc:492] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Bootstrap starting.
I20260927 18:38:01.140395 11243 tablet_bootstrap.cc:492] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: No bootstrap required, opened a new log
I20260927 18:38:01.140610 11243 ts_tablet_manager.cc:1397] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:38:01.141021 11242 tablet_bootstrap.cc:654] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:01.141767 11242 log.cc:824] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:01.142756 11241 raft_consensus.cc:348] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.142948 11241 raft_consensus.cc:374] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:01.143002 11241 raft_consensus.cc:729] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2ec4c64f2b4505975c920599a52330, State: Initialized, Role: FOLLOWER
I20260927 18:38:01.143370 11241 consensus_queue.cc:260] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.143529 11242 tablet_bootstrap.cc:492] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: No bootstrap required, opened a new log
I20260927 18:38:01.143692 11242 ts_tablet_manager.cc:1397] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:38:01.144174 11241 ts_tablet_manager.cc:1428] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 18:38:01.144476 10965 heartbeater.cc:503] Master 127.6.4.62:39963 was elected leader, sending a full tablet report...
I20260927 18:38:01.144553 11243 raft_consensus.cc:348] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.144778 11243 raft_consensus.cc:374] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:01.144826 11243 raft_consensus.cc:729] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b44e999ecd4aacb64bcfbe5e178d87, State: Initialized, Role: FOLLOWER
I20260927 18:38:01.145202 11243 consensus_queue.cc:260] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.146055 11243 ts_tablet_manager.cc:1428] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:38:01.146361 11096 heartbeater.cc:503] Master 127.6.4.62:39963 was elected leader, sending a full tablet report...
I20260927 18:38:01.147524 11242 raft_consensus.cc:348] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.147703 11242 raft_consensus.cc:374] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:01.147759 11242 raft_consensus.cc:729] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff2748df22f45e3b53de26615e62aa2, State: Initialized, Role: FOLLOWER
I20260927 18:38:01.148140 11242 consensus_queue.cc:260] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.148684 11227 heartbeater.cc:503] Master 127.6.4.62:39963 was elected leader, sending a full tablet report...
I20260927 18:38:01.148978 11242 ts_tablet_manager.cc:1428] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260927 18:38:01.182528 11097 tablet.cc:2369] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:01.272168 10966 tablet.cc:2369] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:01.293736 11249 raft_consensus.cc:482] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:01.293932 11249 raft_consensus.cc:504] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.295033 11249 leader_election.cc:288] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2ec4c64f2b4505975c920599a52330 (127.6.4.1:42007), 80b44e999ecd4aacb64bcfbe5e178d87 (127.6.4.2:34311)
I20260927 18:38:01.296854 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.295289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:42007 (local address 127.6.4.3:40659)
0927 18:38:01.295420 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.295434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.295514 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.295857 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.295860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.295874 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:01.296355 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:01.296357 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:01.296367 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:01.296391 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:01.296636 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:01.296639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:01.296642 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:01.296711 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:38:01.296741 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 18:38:01.296882 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.295382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:40659 (local address 127.6.4.1:42007)
0927 18:38:01.295502 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.295507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.295527 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:38:01.295603 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.295606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.295645 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.295736 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.296507 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.296510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.296512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.296521 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.296542 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.296552 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.296636 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.296721 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.296760 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:01.297261 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.295731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:35221 (local address 127.6.4.2:34311)
0927 18:38:01.295875 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.295879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.295980 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:38:01.296113 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.296117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.296151 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.296218 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.296846 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.296850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.296852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.296864 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.296884 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.296894 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.296973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.297113 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.297149 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 18:38:01.297546 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.295546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:34311 (local address 127.6.4.3:35221)
0927 18:38:01.295898 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.295911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.296002 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.296245 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.296248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.296261 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:01.296698 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:01.296700 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:01.296709 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:01.296734 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:01.297011 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:01.297014 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:01.297017 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:38:01.297394 (+ 377us) client_negotiation.cc:240] Negotiation successful
0927 18:38:01.297430 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":130,"threads_started":1}
I20260927 18:38:01.297499 10919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52ffe7848a74d00a94076670df829d7" candidate_uuid: "5ff2748df22f45e3b53de26615e62aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2ec4c64f2b4505975c920599a52330" is_pre_election: true
I20260927 18:38:01.297802 10919 raft_consensus.cc:2454] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff2748df22f45e3b53de26615e62aa2 in term 0.
I20260927 18:38:01.298193 11050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52ffe7848a74d00a94076670df829d7" candidate_uuid: "5ff2748df22f45e3b53de26615e62aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" is_pre_election: true
I20260927 18:38:01.298281 11114 leader_election.cc:302] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2ec4c64f2b4505975c920599a52330, 5ff2748df22f45e3b53de26615e62aa2; no voters:
I20260927 18:38:01.298462 11050 raft_consensus.cc:2454] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff2748df22f45e3b53de26615e62aa2 in term 0.
I20260927 18:38:01.298540 11249 raft_consensus.cc:2781] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:01.298630 11249 raft_consensus.cc:482] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:01.298681 11249 raft_consensus.cc:3037] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:01.299652 11249 raft_consensus.cc:504] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.299922 11249 leader_election.cc:288] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [CANDIDATE]: Term 1 election: Requested vote from peers 2b2ec4c64f2b4505975c920599a52330 (127.6.4.1:42007), 80b44e999ecd4aacb64bcfbe5e178d87 (127.6.4.2:34311)
I20260927 18:38:01.300220 10919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52ffe7848a74d00a94076670df829d7" candidate_uuid: "5ff2748df22f45e3b53de26615e62aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2ec4c64f2b4505975c920599a52330"
I20260927 18:38:01.300299 11050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e52ffe7848a74d00a94076670df829d7" candidate_uuid: "5ff2748df22f45e3b53de26615e62aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80b44e999ecd4aacb64bcfbe5e178d87"
I20260927 18:38:01.300320 10919 raft_consensus.cc:3037] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:01.300420 11050 raft_consensus.cc:3037] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:01.301287 10919 raft_consensus.cc:2454] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff2748df22f45e3b53de26615e62aa2 in term 1.
I20260927 18:38:01.301432 11050 raft_consensus.cc:2454] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff2748df22f45e3b53de26615e62aa2 in term 1.
I20260927 18:38:01.301551 11114 leader_election.cc:302] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2ec4c64f2b4505975c920599a52330, 5ff2748df22f45e3b53de26615e62aa2; no voters:
I20260927 18:38:01.301707 11249 raft_consensus.cc:2781] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:01.301950 11249 raft_consensus.cc:686] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 1 LEADER]: Becoming Leader. State: Replica: 5ff2748df22f45e3b53de26615e62aa2, State: Running, Role: LEADER
I20260927 18:38:01.302254 11249 consensus_queue.cc:237] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } }
I20260927 18:38:01.305263 10777 catalog_manager.cc:5495] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff2748df22f45e3b53de26615e62aa2 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "5ff2748df22f45e3b53de26615e62aa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ff2748df22f45e3b53de26615e62aa2" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:01.313027 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.311653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.6.4.3:37467)
0927 18:38:01.311753 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.311758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.311816 (+ 58us) server_negotiation.cc:407] Connection header received
0927 18:38:01.311969 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.311971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.312021 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.312081 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.312704 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.312708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.312711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.312722 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.312739 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.312748 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.312817 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.312886 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.312921 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:38:01.318116 10919 raft_consensus.cc:1261] T e52ffe7848a74d00a94076670df829d7 P 2b2ec4c64f2b4505975c920599a52330 [term 1 FOLLOWER]: Refusing update from remote peer 5ff2748df22f45e3b53de26615e62aa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:01.318117 11050 raft_consensus.cc:1261] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 1 FOLLOWER]: Refusing update from remote peer 5ff2748df22f45e3b53de26615e62aa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:01.318688 11251 consensus_queue.cc:1035] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2ec4c64f2b4505975c920599a52330" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:01.318830 11249 consensus_queue.cc:1035] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80b44e999ecd4aacb64bcfbe5e178d87" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:01.327518 11255 mvcc.cc:205] Tried to move back new op lower bound from 7334028416273043456 to 7334028416216195072. Current Snapshot: MvccSnapshot[applied={T|T < 7334028416273043456 or (T in {7334028416273043456})}]
I20260927 18:38:01.332906 10776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:01.333029 10776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:01.335116 10776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 471c47dcbab64ae39cf2cfef3ef464ef: Sending DeleteTablet for 3 replicas of tablet e52ffe7848a74d00a94076670df829d7
I20260927 18:38:01.335758 11161 tablet_service.cc:1514] Processing DeleteTablet for tablet e52ffe7848a74d00a94076670df829d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:01 UTC) from {username='slave'} at 127.0.0.1:35924
I20260927 18:38:01.335852 11030 tablet_service.cc:1514] Processing DeleteTablet for tablet e52ffe7848a74d00a94076670df829d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:01 UTC) from {username='slave'} at 127.0.0.1:48794
I20260927 18:38:01.336248 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10837
I20260927 18:38:01.336385 10899 tablet_service.cc:1514] Processing DeleteTablet for tablet e52ffe7848a74d00a94076670df829d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:01 UTC) from {username='slave'} at 127.0.0.1:58510
I20260927 18:38:01.336808 11265 tablet_replica.cc:330] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: stopping tablet replica
I20260927 18:38:01.337100 11265 raft_consensus.cc:2229] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:01.337361 11265 raft_consensus.cc:2258] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:01.337666 11264 tablet_replica.cc:330] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: stopping tablet replica
I20260927 18:38:01.337915 11264 raft_consensus.cc:2229] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:01.338182 11264 raft_consensus.cc:2258] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:01.338961 11265 ts_tablet_manager.cc:1905] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:01.339390 11264 ts_tablet_manager.cc:1905] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:01.341655 11265 ts_tablet_manager.cc:1918] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:01.341806 11265 log.cc:1196] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/e52ffe7848a74d00a94076670df829d7
I20260927 18:38:01.342079 11264 ts_tablet_manager.cc:1918] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:01.342164 11264 log.cc:1196] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/e52ffe7848a74d00a94076670df829d7
I20260927 18:38:01.342264 11265 ts_tablet_manager.cc:1939] T e52ffe7848a74d00a94076670df829d7 P 80b44e999ecd4aacb64bcfbe5e178d87: Deleting consensus metadata
I20260927 18:38:01.342438 11264 ts_tablet_manager.cc:1939] T e52ffe7848a74d00a94076670df829d7 P 5ff2748df22f45e3b53de26615e62aa2: Deleting consensus metadata
I20260927 18:38:01.343389 10762 catalog_manager.cc:4841] TS 5ff2748df22f45e3b53de26615e62aa2 (127.6.4.3:37467): tablet e52ffe7848a74d00a94076670df829d7 (table test-table [id=05804aab655944c587b98689c73d0ba4]) successfully deleted
I20260927 18:38:01.343392 10761 catalog_manager.cc:4841] TS 80b44e999ecd4aacb64bcfbe5e178d87 (127.6.4.2:34311): tablet e52ffe7848a74d00a94076670df829d7 (table test-table [id=05804aab655944c587b98689c73d0ba4]) successfully deleted
W20260927 18:38:01.344468 10761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:42007 (error 108)
I20260927 18:38:01.344511 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10968
I20260927 18:38:01.345690 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.345412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:42007 (local address 127.0.0.1:58518)
0927 18:38:01.345536 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.345614 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:42007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 18:38:01.345827 10761 catalog_manager.cc:4584] TS 2b2ec4c64f2b4505975c920599a52330 (127.6.4.1:42007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e52ffe7848a74d00a94076670df829d7: Network error: Client connection negotiation failed: client connection to 127.6.4.1:42007: connect: Connection refused (error 111)
I20260927 18:38:01.349349 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11099
I20260927 18:38:01.353686 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 10746
2026-09-27T18:38:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (955 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 18:38:01.372812 6160 mini_oidc.cc:152] Starting JWKS server
I20260927 18:38:01.374342 6160 webserver.cc:460] Webserver started at https://127.0.0.1:35433/ using document root <none> and password file <none>
I20260927 18:38:01.374493 6160 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:38:01.374737 6160 webserver.cc:460] Webserver started at http://127.0.0.1:43557/ using document root <none> and password file <none>
2026-09-27T18:38:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:01Z Disabled control of system clock
I20260927 18:38:01.382431 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44765
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:44765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35433/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/testchainca.cert with env {}
W20260927 18:38:01.484359 11272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:01.484579 11272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:01.484615 11272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:01.488610 11272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:01.488668 11272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:01.488683 11272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:01.488698 11272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:01.488723 11272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:38:01.488751 11272 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:38:01.488773 11272 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35433/jwks.json
I20260927 18:38:01.493829 11272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:44765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35433/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:01.494751 11272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:01.495671 11272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:01.503309 11277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:01.503353 11278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:01.503363 11280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:01.503468 11272 server_base.cc:1034] running on GCE node
I20260927 18:38:01.504038 11272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:01.505010 11272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:01.506171 11272 hybrid_clock.cc:648] HybridClock initialized: now 1790534281506151 us; error 30 us; skew 500 ppm
I20260927 18:38:01.507797 11272 webserver.cc:460] Webserver started at http://127.6.4.62:37457/ using document root <none> and password file <none>
I20260927 18:38:01.508338 11272 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:01.508401 11272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:01.508569 11272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:01.510426 11272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "790b0a2e819844739461d70398120b75"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "7abac18a87a32e2b6d7e7f296cd5fb98"
server_key_iv: "b5f9968f04c6653c6ab3f6a3b41a6245"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.510926 11272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "790b0a2e819844739461d70398120b75"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "7abac18a87a32e2b6d7e7f296cd5fb98"
server_key_iv: "b5f9968f04c6653c6ab3f6a3b41a6245"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.514240 11272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:01.516363 11286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:01.517477 11272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:01.517567 11272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "790b0a2e819844739461d70398120b75"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "7abac18a87a32e2b6d7e7f296cd5fb98"
server_key_iv: "b5f9968f04c6653c6ab3f6a3b41a6245"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.517642 11272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:01.546548 11272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:01.547089 11272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:01.547230 11272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:01.553189 11272 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:44765
I20260927 18:38:01.553201 11338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:44765 every 8 connection(s)
I20260927 18:38:01.554015 11272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:01.556923 11339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:01.557219 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11272
I20260927 18:38:01.557315 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:01.557546 6160 external_mini_cluster.cc:1428] Setting key 5090eba0ad8904014754550346ffd1b2
I20260927 18:38:01.563429 11339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75: Bootstrap starting.
I20260927 18:38:01.565862 11339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:01.566612 11339 log.cc:824] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:01.567083 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.558771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.6.4.62:44765)
0927 18:38:01.559380 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.559397 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.559439 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:38:01.560137 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.560161 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.560633 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.561000 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.561397 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.562819 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.563627 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.564295 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.564442 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:01.565111 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.565117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.565131 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.565571 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.565643 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.565668 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.565764 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.565977 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.566263 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":98,"threads_started":1}
I20260927 18:38:01.568605 11339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75: No bootstrap required, opened a new log
I20260927 18:38:01.569752 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:38:01.572329 11339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790b0a2e819844739461d70398120b75" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44765 } }
I20260927 18:38:01.572533 11339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:01.572587 11339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b0a2e819844739461d70398120b75, State: Initialized, Role: FOLLOWER
I20260927 18:38:01.572960 11339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790b0a2e819844739461d70398120b75" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44765 } }
I20260927 18:38:01.573078 11339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:01.573140 11339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:01.573233 11339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:01.574081 11339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790b0a2e819844739461d70398120b75" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44765 } }
I20260927 18:38:01.574378 11339 leader_election.cc:302] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 790b0a2e819844739461d70398120b75; no voters:
I20260927 18:38:01.574640 11339 leader_election.cc:288] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:01.574733 11344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:01.574922 11344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [term 1 LEADER]: Becoming Leader. State: Replica: 790b0a2e819844739461d70398120b75, State: Running, Role: LEADER
I20260927 18:38:01.575201 11344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790b0a2e819844739461d70398120b75" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44765 } }
I20260927 18:38:01.575742 11339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:01.577028 11345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "790b0a2e819844739461d70398120b75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790b0a2e819844739461d70398120b75" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44765 } } }
I20260927 18:38:01.577142 11345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:01.577618 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 790b0a2e819844739461d70398120b75. Latest consensus state: current_term: 1 leader_uuid: "790b0a2e819844739461d70398120b75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "790b0a2e819844739461d70398120b75" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44765 } } }
I20260927 18:38:01.577703 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:01.577943 11355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:01.580470 11355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:01.585706 11355 catalog_manager.cc:1347] Generated new cluster ID: f2e5fc92906245b49946e2ceb6092856
I20260927 18:38:01.585784 11355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:01.611145 11355 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:01.612004 11355 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:01.618741 11355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 790b0a2e819844739461d70398120b75: Generated new TSK 0
I20260927 18:38:01.619314 11355 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 18:38:01.645499 11369 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 18:38:01.645604 11369 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 18:38:01.646081 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.629967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.6.4.62:44765)
0927 18:38:01.630081 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.630086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.630315 (+ 229us) server_negotiation.cc:407] Connection header received
0927 18:38:01.630504 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.630506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.630550 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.630628 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:38:01.630861 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.631369 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.632216 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.632433 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.632514 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:01.632821 (+ 307us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:38:01.645741 (+ 12920us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 18:38:01.645957 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52652: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260927 18:38:01.646157 11342 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52652: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 18:38:01.647074 11368 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:01.629905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44765 (local address 127.0.0.1:52652)
0927 18:38:01.630241 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.630252 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.630367 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.630653 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.630656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.630664 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:38:01.630768 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:01.630773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.631467 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.631471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:01.632118 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:01.632129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.632515 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.632532 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:01.632618 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:01.632657 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:38:01.645911 (+ 13254us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 18:38:01.646105 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.4.62:44765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":89,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.4.62:44765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 18:38:01.647795 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11272
2026-09-27T18:38:01Z chronyd exiting
I20260927 18:38:01.683863 6160 test_util.cc:175] -----------------------------------------------
I20260927 18:38:01.683929 6160 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534257539224-6160-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (312 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 18:38:01.684937 6160 mini_oidc.cc:152] Starting JWKS server
I20260927 18:38:01.685724 6160 webserver.cc:460] Webserver started at https://127.0.0.1:34097/ using document root <none> and password file <none>
I20260927 18:38:01.685822 6160 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:38:01.686062 6160 webserver.cc:460] Webserver started at http://127.0.0.1:46437/ using document root <none> and password file <none>
2026-09-27T18:38:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:01Z Disabled control of system clock
I20260927 18:38:01.693739 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33281
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:42179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:33281
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34097/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/testchainca.cert with env {}
W20260927 18:38:01.794240 11375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:01.794462 11375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:01.794497 11375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:01.798319 11375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:01.798362 11375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:01.798374 11375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:01.798389 11375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:01.798408 11375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:38:01.798424 11375 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:38:01.798439 11375 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34097/jwks.json
I20260927 18:38:01.803383 11375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:42179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:33281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34097/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:01.804253 11375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:01.805112 11375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:01.812624 11380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:01.812618 11381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:01.812674 11375 server_base.cc:1034] running on GCE node
W20260927 18:38:01.812625 11383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:01.813426 11375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:01.814404 11375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:01.815550 11375 hybrid_clock.cc:648] HybridClock initialized: now 1790534281815537 us; error 24 us; skew 500 ppm
I20260927 18:38:01.817224 11375 webserver.cc:460] Webserver started at http://127.6.4.62:36499/ using document root <none> and password file <none>
I20260927 18:38:01.817715 11375 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:01.817770 11375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:01.817943 11375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:01.819806 11375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "01f5a441969840829cf2b48566e36b53"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "d9cfdf3b8b1cdfc13823c69aeabeecaf"
server_key_iv: "c7c6fce37e0016240e5d5239daab307a"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.820320 11375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "01f5a441969840829cf2b48566e36b53"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "d9cfdf3b8b1cdfc13823c69aeabeecaf"
server_key_iv: "c7c6fce37e0016240e5d5239daab307a"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.823470 11375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:38:01.825575 11389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:01.826645 11375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:01.826757 11375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "01f5a441969840829cf2b48566e36b53"
format_stamp: "Formatted at 2026-09-27 18:38:01 on dist-test-slave-bzzj"
server_key: "d9cfdf3b8b1cdfc13823c69aeabeecaf"
server_key_iv: "c7c6fce37e0016240e5d5239daab307a"
server_key_version: "encryptionkey@0"
I20260927 18:38:01.826840 11375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:01.854868 11375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:01.855427 11375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:01.855574 11375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:01.861521 11375 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:33281
I20260927 18:38:01.861541 11441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:33281 every 8 connection(s)
I20260927 18:38:01.862357 11375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:01.865099 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:01.868075 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11375
I20260927 18:38:01.868227 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:01.868539 6160 external_mini_cluster.cc:1428] Setting key f3e5f511a136f5eb1209ecb0c094c685
I20260927 18:38:01.870509 11442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53: Bootstrap starting.
I20260927 18:38:01.872484 11442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:01.873123 11442 log.cc:824] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:01.874693 11442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53: No bootstrap required, opened a new log
I20260927 18:38:01.876914 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.869810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.6.4.62:33281)
0927 18:38:01.870393 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.870407 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.870439 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:38:01.871038 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.871055 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.871422 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.871704 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:01.872123 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.873090 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.873943 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.874388 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.874524 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:01.875269 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:01.875274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:01.875286 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:01.875616 (+ 330us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:01.875667 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:01.875685 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:01.875763 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:01.875888 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 18:38:01.876142 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":119,"threads_started":1}
I20260927 18:38:01.877478 11442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f5a441969840829cf2b48566e36b53" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33281 } }
I20260927 18:38:01.877674 11442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:01.877727 11442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01f5a441969840829cf2b48566e36b53, State: Initialized, Role: FOLLOWER
I20260927 18:38:01.878154 11442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f5a441969840829cf2b48566e36b53" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33281 } }
I20260927 18:38:01.878293 11442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:01.878353 11442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:01.878444 11442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:01.879318 11442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f5a441969840829cf2b48566e36b53" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33281 } }
I20260927 18:38:01.879411 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:38:01.879637 11442 leader_election.cc:302] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01f5a441969840829cf2b48566e36b53; no voters:
I20260927 18:38:01.879916 11442 leader_election.cc:288] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:01.879985 11447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:01.880193 11447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [term 1 LEADER]: Becoming Leader. State: Replica: 01f5a441969840829cf2b48566e36b53, State: Running, Role: LEADER
I20260927 18:38:01.880478 11447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f5a441969840829cf2b48566e36b53" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33281 } }
I20260927 18:38:01.881070 11442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:01.881511 11449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01f5a441969840829cf2b48566e36b53. Latest consensus state: current_term: 1 leader_uuid: "01f5a441969840829cf2b48566e36b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f5a441969840829cf2b48566e36b53" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33281 } } }
I20260927 18:38:01.881642 11449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:01.881904 11448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01f5a441969840829cf2b48566e36b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f5a441969840829cf2b48566e36b53" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33281 } } }
I20260927 18:38:01.881989 11448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:01.882200 11452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:01.885159 11452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:01.889683 11452 catalog_manager.cc:1347] Generated new cluster ID: c3f2fa1874544e2ab74ccb49de78a3fe
I20260927 18:38:01.889740 11452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:01.899226 11452 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:01.899971 11452 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:01.911201 11452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01f5a441969840829cf2b48566e36b53: Generated new TSK 0
I20260927 18:38:01.911785 11452 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 18:38:01.941416 11472 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 18:38:01.941475 11472 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 18:38:01.942073 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:01.926327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.6.4.62:33281)
0927 18:38:01.926441 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:01.926445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:01.926550 (+ 105us) server_negotiation.cc:407] Connection header received
0927 18:38:01.926675 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:01.926678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:01.926728 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:01.926831 (+ 103us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:38:01.927024 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.927842 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.928670 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.929011 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.929105 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:01.929326 (+ 221us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:38:01.941736 (+ 12410us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 18:38:01.941950 (+ 214us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40864: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 18:38:01.942112 11471 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:01.926222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33281 (local address 127.0.0.1:40864)
0927 18:38:01.926476 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:01.926487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:01.926574 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:01.926839 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:01.926842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:01.926848 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:38:01.926936 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:01.926940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.927930 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.927932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:01.928577 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:01.928585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:01.929089 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:01.929103 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:01.929171 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:01.929191 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:38:01.941874 (+ 12683us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 18:38:01.941988 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.4.62:33281: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
W20260927 18:38:01.942129 11445 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40864: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260927 18:38:01.942711 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11375
2026-09-27T18:38:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (295 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 18:38:01.980154 6160 mini_oidc.cc:152] Starting JWKS server
I20260927 18:38:01.980926 6160 webserver.cc:460] Webserver started at https://127.0.0.1:36593/ using document root <none> and password file <none>
I20260927 18:38:01.981007 6160 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:38:01.981240 6160 webserver.cc:460] Webserver started at http://127.0.0.1:34205/ using document root <none> and password file <none>
2026-09-27T18:38:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:01Z Disabled control of system clock
I20260927 18:38:01.988885 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40213
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40213
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36593/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 18:38:02.090313 11478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:02.090530 11478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:02.090580 11478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:02.094408 11478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:02.094455 11478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:02.094477 11478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:02.094496 11478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:02.094516 11478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:38:02.094544 11478 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:38:02.094564 11478 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36593/jwks.json
I20260927 18:38:02.099062 11478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36593/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:02.099948 11478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:02.101154 11478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:02.108793 11483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:02.108786 11484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:02.108770 11486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:02.108944 11478 server_base.cc:1034] running on GCE node
I20260927 18:38:02.109551 11478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:02.110469 11478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:02.111617 11478 hybrid_clock.cc:648] HybridClock initialized: now 1790534282111605 us; error 24 us; skew 500 ppm
I20260927 18:38:02.113231 11478 webserver.cc:460] Webserver started at http://127.6.4.62:43853/ using document root <none> and password file <none>
I20260927 18:38:02.113730 11478 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:02.113791 11478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:02.113966 11478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:02.115767 11478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "9039ad110dc5422a93d8abe3c80e24fc"
format_stamp: "Formatted at 2026-09-27 18:38:02 on dist-test-slave-bzzj"
server_key: "84ec23615b39d7782fe139624aecf09d"
server_key_iv: "ce3b096c8d27d5752b3818a6b3104765"
server_key_version: "encryptionkey@0"
I20260927 18:38:02.116272 11478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "9039ad110dc5422a93d8abe3c80e24fc"
format_stamp: "Formatted at 2026-09-27 18:38:02 on dist-test-slave-bzzj"
server_key: "84ec23615b39d7782fe139624aecf09d"
server_key_iv: "ce3b096c8d27d5752b3818a6b3104765"
server_key_version: "encryptionkey@0"
I20260927 18:38:02.119540 11478 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260927 18:38:02.121762 11492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:02.122888 11478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:02.122991 11478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "9039ad110dc5422a93d8abe3c80e24fc"
format_stamp: "Formatted at 2026-09-27 18:38:02 on dist-test-slave-bzzj"
server_key: "84ec23615b39d7782fe139624aecf09d"
server_key_iv: "ce3b096c8d27d5752b3818a6b3104765"
server_key_version: "encryptionkey@0"
I20260927 18:38:02.123081 11478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:02.132643 11478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:02.133170 11478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:02.133317 11478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:02.139334 11478 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40213
I20260927 18:38:02.139348 11544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40213 every 8 connection(s)
I20260927 18:38:02.140218 11478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:02.143059 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11478
I20260927 18:38:02.143074 11545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:02.143164 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:02.143409 6160 external_mini_cluster.cc:1428] Setting key aec6094b7113fd5205cb134860c6dab7
I20260927 18:38:02.148936 11545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc: Bootstrap starting.
I20260927 18:38:02.151054 11545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:02.151861 11545 log.cc:824] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:02.152391 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:02.144628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.6.4.62:40213)
0927 18:38:02.145221 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:02.145265 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:02.145298 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:38:02.145992 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:02.146008 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:02.146395 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:02.146684 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:02.147201 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.148261 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.149150 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.149609 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.149759 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:02.150560 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:02.150565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:02.150577 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:02.150944 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:02.150997 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:02.151016 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:02.151099 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:02.151247 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 18:38:02.151508 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
I20260927 18:38:02.153743 11545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc: No bootstrap required, opened a new log
I20260927 18:38:02.154917 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:38:02.157401 11545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9039ad110dc5422a93d8abe3c80e24fc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40213 } }
I20260927 18:38:02.157604 11545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:02.157660 11545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9039ad110dc5422a93d8abe3c80e24fc, State: Initialized, Role: FOLLOWER
I20260927 18:38:02.158030 11545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9039ad110dc5422a93d8abe3c80e24fc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40213 } }
I20260927 18:38:02.158156 11545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:02.158218 11545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:02.158313 11545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:02.159116 11545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9039ad110dc5422a93d8abe3c80e24fc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40213 } }
I20260927 18:38:02.159418 11545 leader_election.cc:302] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9039ad110dc5422a93d8abe3c80e24fc; no voters:
I20260927 18:38:02.159705 11545 leader_election.cc:288] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:02.159785 11550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:02.159993 11550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [term 1 LEADER]: Becoming Leader. State: Replica: 9039ad110dc5422a93d8abe3c80e24fc, State: Running, Role: LEADER
I20260927 18:38:02.160286 11550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9039ad110dc5422a93d8abe3c80e24fc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40213 } }
I20260927 18:38:02.160854 11545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:02.161643 11552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9039ad110dc5422a93d8abe3c80e24fc. Latest consensus state: current_term: 1 leader_uuid: "9039ad110dc5422a93d8abe3c80e24fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9039ad110dc5422a93d8abe3c80e24fc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40213 } } }
I20260927 18:38:02.161680 11551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9039ad110dc5422a93d8abe3c80e24fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9039ad110dc5422a93d8abe3c80e24fc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40213 } } }
I20260927 18:38:02.161779 11552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:02.161779 11551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:02.162240 11559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:02.164790 11559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:02.169804 11559 catalog_manager.cc:1347] Generated new cluster ID: 14afae0c59fc4984b41b140c7501fc5f
I20260927 18:38:02.169863 11559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:02.213950 11559 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:02.215310 11559 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:02.222011 11559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9039ad110dc5422a93d8abe3c80e24fc: Generated new TSK 0
I20260927 18:38:02.222824 11559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:38:02.235388 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:02.230722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.6.4.62:40213)
0927 18:38:02.230845 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:02.230849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:02.231012 (+ 163us) server_negotiation.cc:407] Connection header received
0927 18:38:02.231166 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:02.231168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:02.231213 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:02.231302 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:38:02.231517 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.232026 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.233234 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.233443 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.233512 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:02.233806 (+ 294us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:38:02.235024 (+ 1218us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 18:38:02.235246 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57378: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":43}
W20260927 18:38:02.235450 11548 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57378: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260927 18:38:02.237591 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:02.230661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40213 (local address 127.0.0.1:57378)
0927 18:38:02.230931 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:02.230946 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:02.231055 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:02.231314 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:02.231317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:02.231325 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:38:02.231436 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:02.231442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.232148 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.232151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:02.233131 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:02.233141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.233520 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.233541 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:02.233642 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:02.233670 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:38:02.235178 (+ 1508us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 18:38:02.237412 (+ 2234us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.4.62:40213: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":2035,"thread_start_us":102,"threads_started":1}
I20260927 18:38:02.238185 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11478
E20260927 18:38:02.248636 11575 webserver.cc:533] Webserver: sslize failed:
E20260927 18:38:02.248698 11575 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-27T18:38:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T18:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:02Z Disabled control of system clock
I20260927 18:38:02.285683 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33399
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:33399
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/testchainca.cert with env {}
W20260927 18:38:02.394098 11579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:02.394335 11579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:02.394387 11579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:02.398278 11579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:02.398329 11579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:02.398350 11579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:02.398370 11579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:02.398397 11579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:38:02.398420 11579 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:38:02.398444 11579 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 18:38:02.403537 11579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:33399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:02.404495 11579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:02.405398 11579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:02.412918 11587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:02.412921 11585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:02.412945 11584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:02.413137 11579 server_base.cc:1034] running on GCE node
I20260927 18:38:02.413686 11579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:02.414652 11579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:02.415808 11579 hybrid_clock.cc:648] HybridClock initialized: now 1790534282415784 us; error 35 us; skew 500 ppm
I20260927 18:38:02.417480 11579 webserver.cc:460] Webserver started at http://127.6.4.62:42261/ using document root <none> and password file <none>
I20260927 18:38:02.417984 11579 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:02.418044 11579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:02.418212 11579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:02.420151 11579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "30d1705aa66148fe82bff1c66b2f267e"
format_stamp: "Formatted at 2026-09-27 18:38:02 on dist-test-slave-bzzj"
server_key: "0ed5dcae9151d8fabb1e37e466b2552f"
server_key_iv: "0434290109eb993aba4331b04270c385"
server_key_version: "encryptionkey@0"
I20260927 18:38:02.420646 11579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "30d1705aa66148fe82bff1c66b2f267e"
format_stamp: "Formatted at 2026-09-27 18:38:02 on dist-test-slave-bzzj"
server_key: "0ed5dcae9151d8fabb1e37e466b2552f"
server_key_iv: "0434290109eb993aba4331b04270c385"
server_key_version: "encryptionkey@0"
I20260927 18:38:02.423832 11579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260927 18:38:02.426059 11593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:02.427253 11579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:02.427356 11579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "30d1705aa66148fe82bff1c66b2f267e"
format_stamp: "Formatted at 2026-09-27 18:38:02 on dist-test-slave-bzzj"
server_key: "0ed5dcae9151d8fabb1e37e466b2552f"
server_key_iv: "0434290109eb993aba4331b04270c385"
server_key_version: "encryptionkey@0"
I20260927 18:38:02.427441 11579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:02.436349 11579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:02.436890 11579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:02.437032 11579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:02.443156 11579 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:33399
I20260927 18:38:02.443169 11645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:33399 every 8 connection(s)
I20260927 18:38:02.443988 11579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:02.446888 11646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:02.450349 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11579
I20260927 18:38:02.450448 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:02.450691 6160 external_mini_cluster.cc:1428] Setting key 24fff684bb7bf2d091341dce4c987f05
I20260927 18:38:02.452167 11646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e: Bootstrap starting.
I20260927 18:38:02.454223 11646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:02.454883 11646 log.cc:824] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:02.456748 11646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e: No bootstrap required, opened a new log
I20260927 18:38:02.459076 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:02.451951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.6.4.62:33399)
0927 18:38:02.452586 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:02.452601 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:02.452634 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:38:02.453228 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:02.453254 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:02.453636 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:02.453926 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:02.454351 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.455360 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.456202 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.456674 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.456821 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:02.457554 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:02.457559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:02.457571 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:02.457920 (+ 349us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:02.457970 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:02.457988 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:02.458066 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:02.458190 (+ 124us) server_negotiation.cc:299] Negotiation successful
0927 18:38:02.458428 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":146,"threads_started":1}
I20260927 18:38:02.460044 11646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d1705aa66148fe82bff1c66b2f267e" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33399 } }
I20260927 18:38:02.460310 11646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:02.460379 11646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d1705aa66148fe82bff1c66b2f267e, State: Initialized, Role: FOLLOWER
I20260927 18:38:02.460824 11646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d1705aa66148fe82bff1c66b2f267e" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33399 } }
I20260927 18:38:02.460951 11646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:02.461004 11646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:02.461099 11646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:02.461942 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:38:02.462146 11646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d1705aa66148fe82bff1c66b2f267e" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33399 } }
I20260927 18:38:02.462445 11646 leader_election.cc:302] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30d1705aa66148fe82bff1c66b2f267e; no voters:
I20260927 18:38:02.462708 11646 leader_election.cc:288] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:02.462811 11651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:02.462967 11651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [term 1 LEADER]: Becoming Leader. State: Replica: 30d1705aa66148fe82bff1c66b2f267e, State: Running, Role: LEADER
I20260927 18:38:02.463250 11651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d1705aa66148fe82bff1c66b2f267e" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33399 } }
I20260927 18:38:02.463925 11646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:02.464882 11652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30d1705aa66148fe82bff1c66b2f267e. Latest consensus state: current_term: 1 leader_uuid: "30d1705aa66148fe82bff1c66b2f267e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d1705aa66148fe82bff1c66b2f267e" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33399 } } }
I20260927 18:38:02.464977 11653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30d1705aa66148fe82bff1c66b2f267e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d1705aa66148fe82bff1c66b2f267e" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33399 } } }
I20260927 18:38:02.465018 11652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:02.465039 11653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:02.465754 11660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:02.468266 11660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:02.472764 11660 catalog_manager.cc:1347] Generated new cluster ID: f08db91708e14419b4d3a27be9052f7c
I20260927 18:38:02.472826 11660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:02.479755 11660 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:02.480808 11660 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:02.485632 11660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30d1705aa66148fe82bff1c66b2f267e: Generated new TSK 0
I20260927 18:38:02.486161 11660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:38:03.044529 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:02.499656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.6.4.62:33399)
0927 18:38:02.499784 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:02.499789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:02.499985 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:38:02.500152 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:02.500154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:02.500204 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:02.500286 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:38:02.500518 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.501037 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.502235 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.502446 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.502512 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:02.502893 (+ 381us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:38:03.044133 (+541240us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 18:38:03.044401 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44222: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":49}
W20260927 18:38:03.044610 11649 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44222: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 18:38:03.044986 11675 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:02.499606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33399 (local address 127.0.0.1:44222)
0927 18:38:02.499902 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:02.499915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:02.500022 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:02.500303 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:02.500306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:02.500313 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:38:02.500420 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:02.500426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.501140 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.501144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:02.502115 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:02.502124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:02.502519 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:02.502539 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:02.502630 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:02.502658 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:38:03.044361 (+541703us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 18:38:03.044539 (+ 178us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.4.62:33399: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":126,"threads_started":1}
I20260927 18:38:03.045517 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11579
2026-09-27T18:38:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (785 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (118 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (117 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:03 dist-test-slave-bzzj krb5kdc[11687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:03 dist-test-slave-bzzj krb5kdc[11687](info): set up 2 sockets
Sep 27 18:38:03 dist-test-slave-bzzj krb5kdc[11687](info): commencing operation
krb5kdc: starting...
W20260927 18:38:04.224016 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:38:04.334447 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45575
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:45575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:04.438534 11700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:04.438757 11700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:04.438792 11700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:04.442601 11700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:04.442647 11700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:04.442664 11700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:04.442678 11700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:04.442698 11700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:04.447582 11700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:45575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:04.448446 11700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:04.449309 11700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:04.456940 11705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:04.456988 11700 server_base.cc:1034] running on GCE node
W20260927 18:38:04.456925 11706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:04.456945 11708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:04.457655 11700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:04.458647 11700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:04.459811 11700 hybrid_clock.cc:648] HybridClock initialized: now 1790534284459799 us; error 23 us; skew 500 ppm
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:04.462294 11700 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:38:04.463088 11700 webserver.cc:460] Webserver started at http://127.6.4.62:38681/ using document root <none> and password file <none>
I20260927 18:38:04.463563 11700 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:04.463624 11700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:04.463801 11700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:04.465655 11700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "0e00580c0f974f6489ae23937948d18a"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "bc020842bbe55a2814aeae2207f52908"
server_key_iv: "9d49aa3c841c434ddf9aaf4a9bb2f7f5"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.466161 11700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "0e00580c0f974f6489ae23937948d18a"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "bc020842bbe55a2814aeae2207f52908"
server_key_iv: "9d49aa3c841c434ddf9aaf4a9bb2f7f5"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.469503 11700 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260927 18:38:04.471661 11715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.472775 11700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:38:04.472877 11700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "0e00580c0f974f6489ae23937948d18a"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "bc020842bbe55a2814aeae2207f52908"
server_key_iv: "9d49aa3c841c434ddf9aaf4a9bb2f7f5"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.472958 11700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:04.488819 11700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:04.491493 11700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:04.491642 11700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:04.497928 11700 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:45575
I20260927 18:38:04.497938 11767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:45575 every 8 connection(s)
I20260927 18:38:04.498778 11700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:04.499085 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11700
I20260927 18:38:04.499189 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:04.499454 6160 external_mini_cluster.cc:1428] Setting key 9628226891cf70023e8484082ddf0322
I20260927 18:38:04.501536 11768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:04.506611 11768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a: Bootstrap starting.
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:04.508777 11768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:04.509413 11768 log.cc:824] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:04.511427 11768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a: No bootstrap required, opened a new log
I20260927 18:38:04.512045 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:04.500678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.6.4.62:45575)
0927 18:38:04.501090 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:04.501103 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:04.501135 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:38:04.501747 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:04.501764 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:04.502067 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:04.502366 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:04.503102 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.504037 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.504848 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.505282 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.505418 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:04.507684 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:04.507689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:04.507699 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:04.507733 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:04.509216 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:04.509593 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:04.509599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:04.509604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:04.509752 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:04.510037 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:04.510039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:04.510040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:04.510475 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:04.510647 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:04.510874 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 18:38:04.511136 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":132,"threads_started":1}
I20260927 18:38:04.515096 11768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e00580c0f974f6489ae23937948d18a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45575 } }
I20260927 18:38:04.515290 11768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:04.515329 11768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e00580c0f974f6489ae23937948d18a, State: Initialized, Role: FOLLOWER
I20260927 18:38:04.515679 11768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e00580c0f974f6489ae23937948d18a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45575 } }
I20260927 18:38:04.515794 11768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:04.515838 11768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:04.515904 11768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:04.516716 11768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e00580c0f974f6489ae23937948d18a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45575 } }
I20260927 18:38:04.517010 11768 leader_election.cc:302] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e00580c0f974f6489ae23937948d18a; no voters:
I20260927 18:38:04.517239 11768 leader_election.cc:288] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:04.517326 11773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:04.517516 11773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [term 1 LEADER]: Becoming Leader. State: Replica: 0e00580c0f974f6489ae23937948d18a, State: Running, Role: LEADER
I20260927 18:38:04.517833 11773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e00580c0f974f6489ae23937948d18a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45575 } }
I20260927 18:38:04.518225 11768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:04.519196 11774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e00580c0f974f6489ae23937948d18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e00580c0f974f6489ae23937948d18a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45575 } } }
I20260927 18:38:04.519315 11774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:04.519702 11782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:04.519660 11775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e00580c0f974f6489ae23937948d18a. Latest consensus state: current_term: 1 leader_uuid: "0e00580c0f974f6489ae23937948d18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e00580c0f974f6489ae23937948d18a" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45575 } } }
I20260927 18:38:04.519877 11775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:04.522712 11782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:04.527859 11782 catalog_manager.cc:1347] Generated new cluster ID: 698bc57badc84e399c3bb8e36b225b29
I20260927 18:38:04.527907 11782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:04.541392 11782 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:04.542277 11782 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:04.547271 11782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e00580c0f974f6489ae23937948d18a: Generated new TSK 0
I20260927 18:38:04.547842 11782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:38:04.576350 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45575
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:04.704949 11794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:04.705205 11794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:04.705260 11794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:04.709352 11794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:04.709405 11794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:04.709460 11794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:04.714418 11794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:04.715233 11794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:04.716084 11794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:04.724526 11799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:04.724560 11800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:04.724535 11802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:04.724735 11794 server_base.cc:1034] running on GCE node
I20260927 18:38:04.725143 11794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:04.725692 11794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:04.726835 11794 hybrid_clock.cc:648] HybridClock initialized: now 1790534284726813 us; error 34 us; skew 500 ppm
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:04.729295 11794 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:38:04.729990 11794 webserver.cc:460] Webserver started at http://127.6.4.1:43687/ using document root <none> and password file <none>
I20260927 18:38:04.730449 11794 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:04.730501 11794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:04.730643 11794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:04.732471 11794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "ecacb41105ad43d48dc14b95e7428ed1"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "866803fa907d279fad6eceb6e5a1ec72"
server_key_iv: "f3f7a8289b4f4db550660cfc42ff0e06"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.732939 11794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "ecacb41105ad43d48dc14b95e7428ed1"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "866803fa907d279fad6eceb6e5a1ec72"
server_key_iv: "f3f7a8289b4f4db550660cfc42ff0e06"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.736083 11794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:38:04.738155 11809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.739184 11794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 18:38:04.739300 11794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "ecacb41105ad43d48dc14b95e7428ed1"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "866803fa907d279fad6eceb6e5a1ec72"
server_key_iv: "f3f7a8289b4f4db550660cfc42ff0e06"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.739384 11794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:04.763988 11794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:04.766793 11794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:04.766925 11794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:04.767398 11794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:04.768204 11794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:04.768247 11794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.768282 11794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:04.768302 11794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.775873 11794 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:33053
I20260927 18:38:04.775906 11922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:33053 every 8 connection(s)
I20260927 18:38:04.776702 11794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:04.781952 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11794
I20260927 18:38:04.782064 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:04.782359 6160 external_mini_cluster.cc:1428] Setting key ac4229d0ba570db58744e49ccf8bc658
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:04.789350 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:04.778259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:52555 (local address 127.6.4.62:45575)
0927 18:38:04.778395 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:04.778399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:04.779184 (+ 785us) server_negotiation.cc:407] Connection header received
0927 18:38:04.780504 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:04.780506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:04.780551 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:04.780641 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:04.781658 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.782428 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.783632 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.783861 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.783945 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:04.786208 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:04.786211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:04.786213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:04.786235 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:04.787430 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:04.787958 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:04.787960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:04.787961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:04.788031 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:04.788392 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:04.788394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:04.788396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:04.788554 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:04.788624 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:04.789140 (+ 516us) server_negotiation.cc:299] Negotiation successful
0927 18:38:04.789239 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:38:04.790381 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:04.778623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45575 (local address 127.6.4.1:52555)
0927 18:38:04.779046 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:04.779084 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:04.780330 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:04.780796 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:04.780802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:04.781103 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:04.781537 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:04.781549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.782537 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.782540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:04.783535 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:04.783544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.783942 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.783988 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:04.784183 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:04.784647 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:04.784661 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:04.786056 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:04.787537 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:04.787543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:04.787557 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:04.787871 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:04.788127 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:04.788130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:04.788132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:04.788307 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:04.788644 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:04.788651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:04.788925 (+ 274us) client_negotiation.cc:769] Sending connection context
0927 18:38:04.789109 (+ 184us) client_negotiation.cc:240] Negotiation successful
0927 18:38:04.789418 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:38:04.791451 11923 heartbeater.cc:347] Connected to a master server at 127.6.4.62:45575
I20260927 18:38:04.791651 11923 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:04.792155 11923 heartbeater.cc:511] Master 127.6.4.62:45575 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:38:04.793672 11732 ts_manager.cc:194] Registered new tserver with Master: ecacb41105ad43d48dc14b95e7428ed1 (127.6.4.1:33053)
I20260927 18:38:04.794715 11732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:52555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:38:04.807015 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45575
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:04.908593 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:04.908825 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:04.908864 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:04.912726 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:04.912776 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:04.912830 11928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:04.917833 11928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:04.918702 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:04.919669 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:04.928779 11934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:04.928803 11933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:04.928833 11936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:04.929011 11928 server_base.cc:1034] running on GCE node
I20260927 18:38:04.929443 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:04.929986 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:04.931149 11928 hybrid_clock.cc:648] HybridClock initialized: now 1790534284931149 us; error 44 us; skew 500 ppm
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:04.933736 11928 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:38:04.934514 11928 webserver.cc:460] Webserver started at http://127.6.4.2:36351/ using document root <none> and password file <none>
I20260927 18:38:04.935004 11928 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:04.935060 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:04.935252 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:04.937224 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "c1c6422548064ac6aa4b2071854b39c2"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "bae30279313f8ab52f7745ab177878ac"
server_key_iv: "bb14ff7beda95cec59d187fc25af175a"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.937739 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "c1c6422548064ac6aa4b2071854b39c2"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "bae30279313f8ab52f7745ab177878ac"
server_key_iv: "bb14ff7beda95cec59d187fc25af175a"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.941125 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260927 18:38:04.943377 11943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.944559 11928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:04.944655 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "c1c6422548064ac6aa4b2071854b39c2"
format_stamp: "Formatted at 2026-09-27 18:38:04 on dist-test-slave-bzzj"
server_key: "bae30279313f8ab52f7745ab177878ac"
server_key_iv: "bb14ff7beda95cec59d187fc25af175a"
server_key_version: "encryptionkey@0"
I20260927 18:38:04.944751 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:04.968111 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:04.970908 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:04.971063 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:04.971565 11928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:04.972316 11928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:04.972361 11928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.972409 11928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:04.972434 11928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:04.980192 11928 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:39669
I20260927 18:38:04.980227 12056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:39669 every 8 connection(s)
I20260927 18:38:04.980958 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:04.981644 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 11928
I20260927 18:38:04.981753 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:04.982014 6160 external_mini_cluster.cc:1428] Setting key 90c928531b15a09f055d6f813d525286
Sep 27 18:38:04 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534284, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:38:04.994359 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:04.982611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:41869 (local address 127.6.4.62:45575)
0927 18:38:04.982733 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:04.982736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:04.983565 (+ 829us) server_negotiation.cc:407] Connection header received
0927 18:38:04.984936 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:04.984940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:04.984996 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:04.985049 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:04.986402 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.987186 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.988349 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.988696 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.988810 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:04.991143 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:04.991146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:04.991147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:04.991170 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:04.992471 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:04.992958 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:04.992961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:04.992962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:04.993034 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:04.993328 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:04.993331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:04.993333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:04.993546 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:04.993662 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:04.994106 (+ 444us) server_negotiation.cc:299] Negotiation successful
0927 18:38:04.994219 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:04.995317 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:04.982995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45575 (local address 127.6.4.2:41869)
0927 18:38:04.983443 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:04.983483 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:04.984704 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:04.985442 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:04.985452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:04.985832 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:04.986271 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:04.986284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.987302 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.987305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:04.988197 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:04.988206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:04.988796 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:04.988837 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:04.988966 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:04.989467 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:04.989481 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:04.991008 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:04.992573 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:04.992579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:04.992592 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:04.992869 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:04.993121 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:04.993124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:04.993125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:04.993247 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:04.993671 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:04.993675 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:04.993882 (+ 207us) client_negotiation.cc:769] Sending connection context
0927 18:38:04.994047 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 18:38:04.994369 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":116,"threads_started":1}
I20260927 18:38:04.996400 12057 heartbeater.cc:347] Connected to a master server at 127.6.4.62:45575
I20260927 18:38:04.996603 12057 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:04.997035 12057 heartbeater.cc:511] Master 127.6.4.62:45575 requested a full tablet report, sending...
I20260927 18:38:04.997980 11732 ts_manager.cc:194] Registered new tserver with Master: c1c6422548064ac6aa4b2071854b39c2 (127.6.4.2:39669)
I20260927 18:38:04.998495 11732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:41869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:38:05.006916 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45575
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:05.111421 12062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:05.111670 12062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:05.111723 12062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:05.115662 12062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:05.115721 12062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:05.115797 12062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:05.121023 12062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:05.122001 12062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:05.122931 12062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:05.131779 12070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:05.131784 12068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:05.131846 12067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:05.132205 12062 server_base.cc:1034] running on GCE node
I20260927 18:38:05.132555 12062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:05.133112 12062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:05.134261 12062 hybrid_clock.cc:648] HybridClock initialized: now 1790534285134238 us; error 37 us; skew 500 ppm
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.136741 12062 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:38:05.137529 12062 webserver.cc:460] Webserver started at http://127.6.4.3:37921/ using document root <none> and password file <none>
I20260927 18:38:05.138006 12062 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:05.138058 12062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:05.138224 12062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:05.140120 12062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "53f415d7b6c542ad89ce39262d6225cc"
format_stamp: "Formatted at 2026-09-27 18:38:05 on dist-test-slave-bzzj"
server_key: "259f54d59e034e1c10f88c38ab4b248f"
server_key_iv: "634a3ebaf3d8980a2100f77490b3d88b"
server_key_version: "encryptionkey@0"
I20260927 18:38:05.140622 12062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "53f415d7b6c542ad89ce39262d6225cc"
format_stamp: "Formatted at 2026-09-27 18:38:05 on dist-test-slave-bzzj"
server_key: "259f54d59e034e1c10f88c38ab4b248f"
server_key_iv: "634a3ebaf3d8980a2100f77490b3d88b"
server_key_version: "encryptionkey@0"
I20260927 18:38:05.143986 12062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260927 18:38:05.146122 12077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:05.147213 12062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:05.147307 12062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "53f415d7b6c542ad89ce39262d6225cc"
format_stamp: "Formatted at 2026-09-27 18:38:05 on dist-test-slave-bzzj"
server_key: "259f54d59e034e1c10f88c38ab4b248f"
server_key_iv: "634a3ebaf3d8980a2100f77490b3d88b"
server_key_version: "encryptionkey@0"
I20260927 18:38:05.147392 12062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:05.157380 12062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:05.160040 12062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:05.160254 12062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:05.160746 12062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:05.161455 12062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:05.161502 12062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:05.161553 12062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:05.161582 12062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:05.169251 12062 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:42657
I20260927 18:38:05.169275 12190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:42657 every 8 connection(s)
I20260927 18:38:05.170019 12062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:05.172006 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12062
I20260927 18:38:05.172115 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:05.172410 6160 external_mini_cluster.cc:1428] Setting key 0fb57effb42964363ad2a61281610ea5
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:05.187987 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:05.171590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:40893 (local address 127.6.4.62:45575)
0927 18:38:05.171704 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.171708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.172554 (+ 846us) server_negotiation.cc:407] Connection header received
0927 18:38:05.174105 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.174108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.174160 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.174247 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.175344 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.175892 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.177117 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.177333 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.177401 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:05.184740 (+ 7339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:05.184743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:05.184745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:05.184770 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:05.186081 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.186565 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.186567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.186569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.186638 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.186949 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.186952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.186953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.187128 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:05.187205 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:05.187708 (+ 503us) server_negotiation.cc:299] Negotiation successful
0927 18:38:05.187828 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:38:05.188805 12193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:05.171933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45575 (local address 127.6.4.3:40893)
0927 18:38:05.172414 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:05.172466 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:05.173921 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:05.174431 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:05.174440 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:05.174782 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:05.175228 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:05.175240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.175980 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.175983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:05.177018 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:05.177027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.177397 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.177435 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:05.177564 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:05.183094 (+ 5530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:05.183109 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:05.184537 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:05.186194 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:05.186200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:05.186219 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:05.186468 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:05.186719 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:05.186722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:05.186724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:05.186859 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:05.187223 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:05.187230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:05.187474 (+ 244us) client_negotiation.cc:769] Sending connection context
0927 18:38:05.187630 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 18:38:05.187956 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":122,"threads_started":1}
I20260927 18:38:05.189800 12191 heartbeater.cc:347] Connected to a master server at 127.6.4.62:45575
I20260927 18:38:05.190017 12191 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:05.190477 12191 heartbeater.cc:511] Master 127.6.4.62:45575 requested a full tablet report, sending...
I20260927 18:38:05.191368 11732 ts_manager.cc:194] Registered new tserver with Master: 53f415d7b6c542ad89ce39262d6225cc (127.6.4.3:42657)
I20260927 18:38:05.191900 11732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:40893
I20260927 18:38:05.197324 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/test-admin.keytab.
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.212085 6160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 18:38:05.214627 12198 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:05.213404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.6.4.1:33053)
0927 18:38:05.213703 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.213712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.213737 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:38:05.213783 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.213788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.213967 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.214082 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.214481 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41928: BlockingRecv error: recv got EOF from 127.0.0.1:41928 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
W20260927 18:38:05.214895 6160 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc)
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.215785 6160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.216706 6160 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:38:05.225788 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:05.217469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.6.4.1:33053)
0927 18:38:05.217574 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.217578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.217667 (+ 89us) server_negotiation.cc:407] Connection header received
0927 18:38:05.217785 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.217788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.217824 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.217885 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.218367 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.219374 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.220330 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.220713 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.220812 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:05.222785 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:05.222794 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:05.222808 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:05.222830 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:05.224262 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.224665 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.224670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.224672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.224746 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.224959 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.224961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.224962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.225172 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:05.225378 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:05.225503 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 18:38:05.225657 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 18:38:05.229157 12198 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:05.228428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.6.4.1:33053)
0927 18:38:05.228534 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.228537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.228553 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:38:05.228700 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.228702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.228734 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.228789 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.229074 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41960: BlockingRecv error: recv got EOF from 127.0.0.1:41960 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260927 18:38:05.229257 6160 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc' of type FILE
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.230114 6160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.230878 6160 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:38:05.238709 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:05.231621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.6.4.1:33053)
0927 18:38:05.231813 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.231816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.231826 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:05.231852 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.231854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.231882 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.231940 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.232423 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.232926 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.233633 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.233844 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.233913 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:05.235710 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:05.235712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:05.235714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:05.235733 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:05.237110 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.237546 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.237550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.237551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.237650 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.237901 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.237905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.237907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.238063 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:05.238179 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:05.238476 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 18:38:05.238571 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
W20260927 18:38:05.240835 12198 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:05.240159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.6.4.1:33053)
0927 18:38:05.240251 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.240254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.240299 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:38:05.240390 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.240392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.240421 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.240476 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.240746 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41974: BlockingRecv error: recv got EOF from 127.0.0.1:41974 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 18:38:05.240908 6160 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534257539224-6160-0/krb5kdc/krb5cc)
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.241776 6160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:05.242537 6160 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[11687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:38:05.249508 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:05.243193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.6.4.1:33053)
0927 18:38:05.243280 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:05.243284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:05.243349 (+ 65us) server_negotiation.cc:407] Connection header received
0927 18:38:05.243428 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:05.243430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:05.243458 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:05.243515 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:05.243895 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.244487 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.245193 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:05.245400 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:05.245471 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:05.247241 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:05.247244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:05.247246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:05.247264 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:05.248387 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.248734 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.248736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.248737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.248801 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:05.248967 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:05.248969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:05.248970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:05.249097 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:05.249177 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:05.249315 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 18:38:05.249386 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:38:05.250358 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11794
I20260927 18:38:05.255692 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11928
I20260927 18:38:05.259685 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12062
I20260927 18:38:05.263783 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 11700
2026-09-27T18:38:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1984 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[12211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[12211](info): set up 2 sockets
Sep 27 18:38:05 dist-test-slave-bzzj krb5kdc[12211](info): commencing operation
krb5kdc: starting...
W20260927 18:38:06.220126 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab.
Entry for principal oryx/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab.
I20260927 18:38:06.328971 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:43091
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab
--principal=oryx/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:43091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:06.440807 12224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:06.441041 12224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:06.441078 12224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:06.444948 12224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:06.444993 12224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:06.445010 12224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:06.445024 12224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:06.445037 12224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:06.450023 12224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:43091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab
--principal=oryx/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:43091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:06.450905 12224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:06.451792 12224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:06.459643 12229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:06.459673 12232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:06.459674 12230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:06.459934 12224 server_base.cc:1034] running on GCE node
I20260927 18:38:06.460469 12224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:06.461472 12224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:06.462618 12224 hybrid_clock.cc:648] HybridClock initialized: now 1790534286462596 us; error 40 us; skew 500 ppm
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:06.465108 12224 init.cc:377] Logged in from keytab as oryx/127.6.4.62@KRBTEST.COM (short username oryx)
I20260927 18:38:06.465929 12224 webserver.cc:460] Webserver started at http://127.6.4.62:40085/ using document root <none> and password file <none>
I20260927 18:38:06.466452 12224 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:06.466512 12224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:06.466691 12224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:06.468638 12224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "18fff560805b45478e311f545c5fbede"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "0137e8c6c20c5ec43b706020a3e109fa"
server_key_iv: "3189afa1bede1235a0261d564b5f41bd"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.469240 12224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "18fff560805b45478e311f545c5fbede"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "0137e8c6c20c5ec43b706020a3e109fa"
server_key_iv: "3189afa1bede1235a0261d564b5f41bd"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.472659 12224 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:38:06.474879 12239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.475967 12224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:06.476071 12224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "18fff560805b45478e311f545c5fbede"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "0137e8c6c20c5ec43b706020a3e109fa"
server_key_iv: "3189afa1bede1235a0261d564b5f41bd"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.476187 12224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:06.491959 12224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:06.494845 12224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:06.495008 12224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:06.501426 12224 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:43091
I20260927 18:38:06.501442 12291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:43091 every 8 connection(s)
I20260927 18:38:06.502292 12224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:06.503662 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12224
I20260927 18:38:06.503787 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:06.504084 6160 external_mini_cluster.cc:1428] Setting key 2b1dc2ece82674ee115a4a0a89cb23d0
I20260927 18:38:06.505443 12292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:06.511895 12292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Bootstrap starting.
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:06.514320 12292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:06.514990 12292 log.cc:824] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:06.516990 12292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: No bootstrap required, opened a new log
I20260927 18:38:06.517730 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:06.505360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.6.4.62:43091)
0927 18:38:06.505734 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:06.505752 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:06.505792 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:38:06.506613 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:06.506634 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:06.507031 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:06.507389 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:06.508025 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.509513 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.510300 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.510974 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.511137 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:06.513649 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:06.513654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:06.513666 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:06.513692 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:06.515223 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:06.515619 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:06.515625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:06.515630 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:06.515735 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:06.515996 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:06.515998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:06.515999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:06.516399 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:06.516530 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:06.516799 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 18:38:06.517037 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260927 18:38:06.520630 12292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } }
I20260927 18:38:06.520833 12292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:06.520888 12292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fff560805b45478e311f545c5fbede, State: Initialized, Role: FOLLOWER
I20260927 18:38:06.521275 12292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } }
I20260927 18:38:06.521395 12292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:06.521457 12292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:06.521548 12292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:06.522317 12292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } }
I20260927 18:38:06.522615 12292 leader_election.cc:302] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18fff560805b45478e311f545c5fbede; no voters:
I20260927 18:38:06.522862 12292 leader_election.cc:288] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:06.522953 12297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:06.523161 12297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 LEADER]: Becoming Leader. State: Replica: 18fff560805b45478e311f545c5fbede, State: Running, Role: LEADER
I20260927 18:38:06.523483 12297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } }
I20260927 18:38:06.523892 12292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:06.524910 12298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } }
I20260927 18:38:06.525029 12298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:06.525384 12306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:06.525359 12299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18fff560805b45478e311f545c5fbede. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } }
I20260927 18:38:06.525486 12299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:06.528065 12306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:06.532686 12306 catalog_manager.cc:1347] Generated new cluster ID: 5cc6d271cd784999a345512aad417549
I20260927 18:38:06.532745 12306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:06.545923 12306 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:06.546762 12306 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:06.554772 12306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Generated new TSK 0
I20260927 18:38:06.555354 12306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab.
Entry for principal oryx/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab.
I20260927 18:38:06.585018 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:43091
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:06.688580 12318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:06.688793 12318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:06.688831 12318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:06.692695 12318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:06.692739 12318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:06.692796 12318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:06.697671 12318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:43091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:06.698488 12318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:06.699352 12318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:06.707930 12324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:06.708053 12323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:06.707945 12326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:06.708130 12318 server_base.cc:1034] running on GCE node
I20260927 18:38:06.708688 12318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:06.709205 12318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:06.710354 12318 hybrid_clock.cc:648] HybridClock initialized: now 1790534286710333 us; error 35 us; skew 500 ppm
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:06.712702 12318 init.cc:377] Logged in from keytab as oryx/127.6.4.1@KRBTEST.COM (short username oryx)
I20260927 18:38:06.713420 12318 webserver.cc:460] Webserver started at http://127.6.4.1:41833/ using document root <none> and password file <none>
I20260927 18:38:06.713894 12318 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:06.713954 12318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:06.714113 12318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:06.715855 12318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "d8079d590210424dbdeac88bbe706045"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "14410e20a0cfcd2847c1157535674755"
server_key_iv: "1d869f14e4ad0a7d755e2be8d30dd965"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.716355 12318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "d8079d590210424dbdeac88bbe706045"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "14410e20a0cfcd2847c1157535674755"
server_key_iv: "1d869f14e4ad0a7d755e2be8d30dd965"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.719451 12318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:38:06.721563 12333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.722620 12318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:06.722723 12318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "d8079d590210424dbdeac88bbe706045"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "14410e20a0cfcd2847c1157535674755"
server_key_iv: "1d869f14e4ad0a7d755e2be8d30dd965"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.722805 12318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:06.737231 12318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:06.739599 12318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:06.739749 12318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:06.740238 12318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:06.740967 12318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:06.741009 12318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.741056 12318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:06.741086 12318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.748883 12318 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:34467
I20260927 18:38:06.748955 12446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:34467 every 8 connection(s)
I20260927 18:38:06.749667 12318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:06.749876 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12318
I20260927 18:38:06.749972 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:06.750274 6160 external_mini_cluster.cc:1428] Setting key 3e6b240a8ae5e7026deb3f5f1f4d6d7f
WARNING: no policy specified for oryx/127.6.4.2@KRBTEST.COM; defaulting to no policy
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.2@KRBTEST.COM" created.
I20260927 18:38:06.762895 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:06.751243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:45921 (local address 127.6.4.62:43091)
0927 18:38:06.751373 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:06.751378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:06.752254 (+ 876us) server_negotiation.cc:407] Connection header received
0927 18:38:06.753659 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:06.753662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:06.753720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:06.753788 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:06.754826 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.755405 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.757109 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.757354 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.757447 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:06.759819 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:06.759822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:06.759824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:06.759848 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:06.761117 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:06.761526 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:06.761528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:06.761530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:06.761611 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:06.761928 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:06.761930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:06.761931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:06.762092 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:06.762177 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:06.762617 (+ 440us) server_negotiation.cc:299] Negotiation successful
0927 18:38:06.762733 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:06.763746 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:06.751625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:45921)
0927 18:38:06.752086 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:06.752159 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:06.753464 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:06.753947 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:06.753954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:06.754276 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:06.754697 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:06.754710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.755557 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.755561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:06.756967 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:06.756980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.757452 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.757505 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:06.757699 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:06.758231 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:06.758250 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:06.759669 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:06.761223 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:06.761227 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:06.761237 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:06.761455 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:06.761687 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:06.761690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:06.761692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:06.761850 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:06.762190 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:06.762197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:06.762409 (+ 212us) client_negotiation.cc:769] Sending connection context
0927 18:38:06.762560 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 18:38:06.762857 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":109,"threads_started":1}
I20260927 18:38:06.764837 12447 heartbeater.cc:347] Connected to a master server at 127.6.4.62:43091
I20260927 18:38:06.765038 12447 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:06.765442 12447 heartbeater.cc:511] Master 127.6.4.62:43091 requested a full tablet report, sending...
I20260927 18:38:06.766776 12256 ts_manager.cc:194] Registered new tserver with Master: d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467)
I20260927 18:38:06.767802 12256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:45921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab.
Entry for principal oryx/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab.
I20260927 18:38:06.774906 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:43091
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:06.876005 12452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:06.876235 12452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:06.876276 12452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:06.880221 12452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:06.880265 12452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:06.880328 12452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:06.885325 12452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:43091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:06.886195 12452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:06.887081 12452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:06.895702 12458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:06.895680 12460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:06.895710 12457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:06.895970 12452 server_base.cc:1034] running on GCE node
I20260927 18:38:06.896345 12452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:06.896854 12452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:06.898001 12452 hybrid_clock.cc:648] HybridClock initialized: now 1790534286897988 us; error 23 us; skew 500 ppm
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:06.900408 12452 init.cc:377] Logged in from keytab as oryx/127.6.4.2@KRBTEST.COM (short username oryx)
I20260927 18:38:06.901140 12452 webserver.cc:460] Webserver started at http://127.6.4.2:33543/ using document root <none> and password file <none>
I20260927 18:38:06.901623 12452 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:06.901681 12452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:06.901844 12452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:06.903651 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "0a428c09443549899f21249cf969ff9c"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "f933199d0c5f0f85a2f893c73d189909"
server_key_iv: "76b075778a104d9f1d3b5da673363ada"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.904189 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "0a428c09443549899f21249cf969ff9c"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "f933199d0c5f0f85a2f893c73d189909"
server_key_iv: "76b075778a104d9f1d3b5da673363ada"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.907440 12452 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260927 18:38:06.909534 12467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.910604 12452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:06.910699 12452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "0a428c09443549899f21249cf969ff9c"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "f933199d0c5f0f85a2f893c73d189909"
server_key_iv: "76b075778a104d9f1d3b5da673363ada"
server_key_version: "encryptionkey@0"
I20260927 18:38:06.910779 12452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:06.936708 12452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:06.939117 12452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:06.939280 12452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:06.939800 12452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:06.940594 12452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:06.940639 12452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.940686 12452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:06.940707 12452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:06.948381 12452 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:33529
I20260927 18:38:06.948422 12580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:33529 every 8 connection(s)
I20260927 18:38:06.949158 12452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:06.949685 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12452
I20260927 18:38:06.949805 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:06.950079 6160 external_mini_cluster.cc:1428] Setting key d31933b7267525af88d2b9ed1732b323
Sep 27 18:38:06 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.3@KRBTEST.COM" created.
I20260927 18:38:06.961967 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:06.950727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:38365 (local address 127.6.4.62:43091)
0927 18:38:06.952069 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:06.952073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:06.952088 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:38:06.953111 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:06.953114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:06.953157 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:06.953222 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:06.954581 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.955110 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.956439 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.956653 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.956738 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:06.958954 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:06.958957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:06.958959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:06.958983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:06.960069 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:06.960573 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:06.960576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:06.960577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:06.960646 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:06.961007 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:06.961010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:06.961012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:06.961179 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:06.961273 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:06.961696 (+ 423us) server_negotiation.cc:299] Negotiation successful
0927 18:38:06.961810 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1256}
I20260927 18:38:06.962718 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:06.951174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:38365)
0927 18:38:06.951610 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:06.951659 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:06.952902 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:06.953395 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:06.953404 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:06.953878 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:06.954457 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:06.954474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.955237 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.955240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:06.956329 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:06.956338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:06.956735 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:06.956796 (+ 61us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:06.956947 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:06.957472 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:06.957486 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:06.958812 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:06.960176 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:06.960181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:06.960195 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:06.960484 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:06.960777 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:06.960780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:06.960782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:06.960899 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:06.961273 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:06.961278 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:06.961487 (+ 209us) client_negotiation.cc:769] Sending connection context
0927 18:38:06.961638 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 18:38:06.961931 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":92,"threads_started":1}
I20260927 18:38:06.963793 12581 heartbeater.cc:347] Connected to a master server at 127.6.4.62:43091
I20260927 18:38:06.963981 12581 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:06.964413 12581 heartbeater.cc:511] Master 127.6.4.62:43091 requested a full tablet report, sending...
I20260927 18:38:06.965322 12256 ts_manager.cc:194] Registered new tserver with Master: 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:06.965827 12256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:38365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab.
Entry for principal oryx/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab.
I20260927 18:38:06.972940 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:43091
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:07.074927 12586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:07.075141 12586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:07.075179 12586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:07.079088 12586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:07.079144 12586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:07.079200 12586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:07.084236 12586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:43091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:07.085144 12586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:07.086055 12586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:07.095033 12592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.095047 12594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.095047 12591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.095239 12586 server_base.cc:1034] running on GCE node
I20260927 18:38:07.095634 12586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:07.096254 12586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:07.097404 12586 hybrid_clock.cc:648] HybridClock initialized: now 1790534287097390 us; error 24 us; skew 500 ppm
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:07.099911 12586 init.cc:377] Logged in from keytab as oryx/127.6.4.3@KRBTEST.COM (short username oryx)
I20260927 18:38:07.100710 12586 webserver.cc:460] Webserver started at http://127.6.4.3:38027/ using document root <none> and password file <none>
I20260927 18:38:07.101199 12586 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.101260 12586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.101428 12586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:07.103338 12586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "41d9d386cbd44064877578ce2fe3c0e0"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "794da5e4a98b13d8f0a96c7101ecf212"
server_key_iv: "8f4c10bd5021a8f8d67a5fa805d0c423"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.103852 12586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "41d9d386cbd44064877578ce2fe3c0e0"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "794da5e4a98b13d8f0a96c7101ecf212"
server_key_iv: "8f4c10bd5021a8f8d67a5fa805d0c423"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.107276 12586 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 18:38:07.109467 12601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.110543 12586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:07.110639 12586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "41d9d386cbd44064877578ce2fe3c0e0"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "794da5e4a98b13d8f0a96c7101ecf212"
server_key_iv: "8f4c10bd5021a8f8d67a5fa805d0c423"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.110745 12586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.128826 12586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:07.131791 12586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:07.131960 12586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:07.132543 12586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:07.133351 12586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:07.133399 12586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.133450 12586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:07.133478 12586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.141868 12586 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:34573
I20260927 18:38:07.141901 12714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:34573 every 8 connection(s)
I20260927 18:38:07.142690 12586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:07.147802 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12586
I20260927 18:38:07.147891 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:07.148175 6160 external_mini_cluster.cc:1428] Setting key 53678fce83a139f2da83465b2bc6d838
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:07.155107 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.144417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:53773 (local address 127.6.4.62:43091)
0927 18:38:07.144539 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.144542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.145333 (+ 791us) server_negotiation.cc:407] Connection header received
0927 18:38:07.146499 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.146503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.146550 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.146622 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.147714 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.148377 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.149628 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.149966 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.150044 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.152223 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.152227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.152229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.152252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.153362 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.153879 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.153881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.153882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.153953 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.154223 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.154225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.154226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.154384 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.154454 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.154899 (+ 445us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.154990 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:38:07.155871 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.144782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:53773)
0927 18:38:07.145193 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.145233 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.146295 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.146769 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.146775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.147098 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.147573 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.147591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.148482 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.148485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.149532 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.149541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.150050 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.150105 (+ 55us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.150255 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.150774 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.150789 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.152060 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.153470 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.153475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.153485 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.153798 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.154015 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.154018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.154019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.154149 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.154488 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.154493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.154719 (+ 226us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.154869 (+ 150us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.155096 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":105,"threads_started":1}
I20260927 18:38:07.157033 12715 heartbeater.cc:347] Connected to a master server at 127.6.4.62:43091
I20260927 18:38:07.157271 12715 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:07.157742 12715 heartbeater.cc:511] Master 127.6.4.62:43091 requested a full tablet report, sending...
I20260927 18:38:07.158715 12256 ts_manager.cc:194] Registered new tserver with Master: 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573)
I20260927 18:38:07.159247 12256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:53773
I20260927 18:38:07.161670 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.4.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.61.keytab.
Entry for principal oryx/127.6.4.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.61.keytab.
I20260927 18:38:07.186779 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:42643
--webserver_interface=127.6.4.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.61.keytab
--principal=oryx/127.6.4.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:43091,127.6.4.61:42643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:07.291935 12720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:07.292215 12720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:07.292281 12720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:07.296137 12720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:07.296186 12720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:07.296221 12720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:07.296243 12720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:07.296271 12720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:07.301256 12720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:43091,127.6.4.61:42643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.61.keytab
--principal=oryx/127.6.4.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:42643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.4.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
W20260927 18:38:07.301440 12720 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.4.62:43091,127.6.4.61:42643'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 18:38:07.302209 12720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:07.303123 12720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:07.310910 12725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.311031 12726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.310971 12720 server_base.cc:1034] running on GCE node
W20260927 18:38:07.310930 12728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.311621 12720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:07.312666 12720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:07.313815 12720 hybrid_clock.cc:648] HybridClock initialized: now 1790534287313798 us; error 28 us; skew 500 ppm
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:07.316385 12720 init.cc:377] Logged in from keytab as oryx/127.6.4.61@KRBTEST.COM (short username oryx)
I20260927 18:38:07.317155 12720 webserver.cc:460] Webserver started at http://127.6.4.61:45693/ using document root <none> and password file <none>
I20260927 18:38:07.317637 12720 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.317689 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.317847 12720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:07.319687 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/instance:
uuid: "9eacdc885dfb4f8d94656268bf781c60"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7b995da6eb534add0f376d9a0193a318"
server_key_iv: "9401d9c40afbe068c9a21109b11c5529"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.320196 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal/instance:
uuid: "9eacdc885dfb4f8d94656268bf781c60"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7b995da6eb534add0f376d9a0193a318"
server_key_iv: "9401d9c40afbe068c9a21109b11c5529"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.323477 12720 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:38:07.325659 12735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.326800 12720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:07.326901 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
uuid: "9eacdc885dfb4f8d94656268bf781c60"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7b995da6eb534add0f376d9a0193a318"
server_key_iv: "9401d9c40afbe068c9a21109b11c5529"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.327014 12720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.341269 12720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:07.343776 12720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:07.343927 12720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:07.349819 12720 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.61:42643
I20260927 18:38:07.350060 12787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.61:42643 every 8 connection(s)
I20260927 18:38:07.350669 12720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
I20260927 18:38:07.351513 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12720
I20260927 18:38:07.353497 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:07.357717 12788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.61@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:07.365447 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.352809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48680 (local address 127.6.4.61:42643)
0927 18:38:07.353168 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.353186 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.353226 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:38:07.354038 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.354058 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.354439 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.354774 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.355377 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.356916 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.357716 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.358358 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.358501 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.361167 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.361171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.361181 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.361205 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.362860 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.363223 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.363228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.363234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.363335 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.363566 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.363569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.363571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.363897 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.364059 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.364339 (+ 280us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.364591 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":99,"threads_started":1}
I20260927 18:38:07.367643 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.359499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.6.4.62:43091)
0927 18:38:07.359603 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.359607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.359932 (+ 325us) server_negotiation.cc:407] Connection header received
0927 18:38:07.360059 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.360063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.360140 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.360216 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.360950 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.361469 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.362665 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.362869 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.362957 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.364834 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.364838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.364841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.364868 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.366123 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.366526 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.366529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.366530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.366603 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.366829 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.366832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.366834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.367026 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.367110 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.367402 (+ 292us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.367503 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:38:07.367653 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.359472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56388)
0927 18:38:07.359821 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.359854 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.359957 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.360276 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.360282 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.360689 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.360841 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.360849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.361583 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.361587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.362560 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.362570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.362965 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.362979 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.363081 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.363480 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.363490 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.364709 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.366218 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.366225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.366229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.366435 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.366682 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.366685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.366686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.366752 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.367119 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.367125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.367214 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.367332 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.367482 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":138,"threads_started":1}
I20260927 18:38:07.368754 12788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.61@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
I20260927 18:38:07.375305 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.369094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48690)
0927 18:38:07.369160 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.369169 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.369232 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.369411 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.369413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.369534 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.369601 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.369605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.370575 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.370578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.371048 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.371055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.371449 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.371462 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.371538 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.371836 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.371846 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.372867 (+ 1021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.374196 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.374198 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.374200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.374350 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.374569 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.374572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.374573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.374658 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.374942 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.374945 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.375012 (+ 67us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.375087 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.375186 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260927 18:38:07.375372 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.369178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.6.4.61:42643)
0927 18:38:07.369260 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.369263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.369276 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:07.369302 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.369304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.369334 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.369393 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.369690 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.370483 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.371140 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.371359 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.371446 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.372967 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.372970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.372972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.372991 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.374114 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.374413 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.374415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.374416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.374473 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.374734 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.374736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.374737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.374863 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.374924 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.375119 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.375222 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":32}
I20260927 18:38:07.377588 12788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Bootstrap starting.
I20260927 18:38:07.379608 12788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:07.380292 12788 log.cc:824] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:07.381845 12788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: No bootstrap required, opened a new log
I20260927 18:38:07.384678 12788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } }
I20260927 18:38:07.384847 12788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:07.384886 12788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eacdc885dfb4f8d94656268bf781c60, State: Initialized, Role: FOLLOWER
I20260927 18:38:07.385226 12788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } }
I20260927 18:38:07.385567 12794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } } }
I20260927 18:38:07.385672 12794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:07.386205 12788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 18:38:07.390434 12805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:38:07.390537 12805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:38:07.391798 12720 master_runner.cc:428] Detected that this master 9eacdc885dfb4f8d94656268bf781c60 is joining an existing cluster
I20260927 18:38:07.391845 12720 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.4.61:42643
I20260927 18:38:07.401377 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.393868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48706)
0927 18:38:07.394092 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.394105 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.394211 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.394421 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.394424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.394681 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.394811 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.394818 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.395662 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.395666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.396388 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.396396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.397251 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.397262 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.397326 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.397777 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.397787 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.398138 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.399840 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.399843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.399845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.400131 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.400383 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.400386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.400388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.400485 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.400844 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.400847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.400929 (+ 82us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.401028 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.401153 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":58,"spinlock_wait_cycles":1920,"thread_start_us":89,"threads_started":1}
I20260927 18:38:07.401386 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.393681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.6.4.62:43091)
0927 18:38:07.393801 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.393805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.393952 (+ 147us) server_negotiation.cc:407] Connection header received
0927 18:38:07.394094 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.394097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.394144 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.394209 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.394725 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.395488 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.396854 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.397093 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.397164 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.398442 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.398446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.398449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.398473 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.399847 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.400279 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.400282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.400284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.400358 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.400617 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.400620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.400622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.400812 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.400871 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.401154 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.401250 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:38:07.401386 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.393689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.6.4.61:42643)
0927 18:38:07.393788 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.393792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.394180 (+ 388us) server_negotiation.cc:407] Connection header received
0927 18:38:07.394283 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.394285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.394320 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.394390 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.394916 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.395593 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.396481 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.396757 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.396845 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.398248 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.398252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.398255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.398276 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.399731 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.400223 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.400226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.400228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.400303 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.400574 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.400577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.400579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.400756 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.400848 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.401080 (+ 232us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.401223 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":41}
I20260927 18:38:07.401791 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.393618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56392)
0927 18:38:07.393863 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.393875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.393971 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.394312 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.394315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.394538 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.394623 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.394628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.395610 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.395614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.396726 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.396737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.397193 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.397208 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.397296 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.397844 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.397858 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.398303 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.399917 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.399920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.399922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.400175 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.400454 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.400456 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.400457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.400526 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.400933 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.400935 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.400996 (+ 61us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.401511 (+ 515us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.401627 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":56,"spinlock_wait_cycles":1792,"thread_start_us":111,"threads_started":1}
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.62@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
I20260927 18:38:07.413053 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.405533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.6.4.61:42643)
0927 18:38:07.405622 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.405626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.405889 (+ 263us) server_negotiation.cc:407] Connection header received
0927 18:38:07.406050 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.406053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.406096 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.406155 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.406722 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.407168 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.407967 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.408227 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.408306 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.410380 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.410382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.410384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.410403 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.411568 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.411970 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.411974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.411975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.412041 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.412295 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.412299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.412300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.412445 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.412522 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.412826 (+ 304us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.412917 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:38:07.413077 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.405501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48720)
0927 18:38:07.405785 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.405823 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.405940 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.406197 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.406221 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.406523 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.406613 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.406619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.407261 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.407264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.407865 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.407873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.408305 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.408319 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.408417 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.408860 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.408871 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.410264 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.411663 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.411671 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.411676 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.411886 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.412137 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.412140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.412142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.412220 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.412536 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.412542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.412644 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.412764 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.412926 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260927 18:38:07.414191 12256 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.4.61:42643: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60"
member_type: NON_VOTER
last_known_addr {
host: "127.6.4.61"
port: 42643
}
attrs {
promote: true
}
}
dest_uuid: "18fff560805b45478e311f545c5fbede"
cas_config_opid_index: -1
I20260927 18:38:07.414775 12256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } }
I20260927 18:38:07.416181 12815 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } }
I20260927 18:38:07.416953 12762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:07.417737 12815 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.4.61:42643
I20260927 18:38:07.418141 12720 master.cc:537] Master@127.6.4.61:42643 shutting down...
I20260927 18:38:07.418241 12762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Refusing update from remote peer 18fff560805b45478e311f545c5fbede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 18:38:07.419227 12815 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:07.419631 12814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } }
I20260927 18:38:07.419798 12814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
W20260927 18:38:07.421283 12241 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260927 18:38:07.422086 12241 consensus_peers.cc:487] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede -> Peer 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61:42643): Couldn't send request to peer 9eacdc885dfb4f8d94656268bf781c60. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:38:07.423017 12720 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:07.423269 12720 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:07.423372 12720 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: stopping tablet replica
W20260927 18:38:07.425276 12736 connection.cc:441] server connection from 127.0.0.1:48680 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:48680 (request call id 13) could send its response
I20260927 18:38:07.437994 12720 master.cc:552] Master@127.6.4.61:42643 shutdown complete.
I20260927 18:38:07.438042 12720 master_runner.cc:305] Clearing existing system tablet
I20260927 18:38:07.438269 12720 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.438319 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.439451 12720 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:38:07.440219 12819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.440403 12720 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.440506 12720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:07.440536 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
uuid: "9eacdc885dfb4f8d94656268bf781c60"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7b995da6eb534add0f376d9a0193a318"
server_key_iv: "9401d9c40afbe068c9a21109b11c5529"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.442081 12720 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:07.443780 12720 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 18:38:07.443847 12720 log.cc:1196] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 18:38:07.444044 12720 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Deleting consensus metadata
I20260927 18:38:07.444219 12720 master_runner.cc:315] Copying system tablet from 127.6.4.62:43091
I20260927 18:38:07.445143 12720 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: tablet copy: Beginning tablet copy session from remote peer at address 127.6.4.62:43091
I20260927 18:38:07.452322 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.445573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56394)
0927 18:38:07.445786 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.445800 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.445893 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.446100 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.446102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.446269 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.446360 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.446366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.447342 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.447345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.447916 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.447923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.448461 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.448474 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.448543 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.449009 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.449020 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.449328 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.451015 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.451018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.451019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.451199 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.451463 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.451465 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.451466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.451530 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.451884 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.451886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.451958 (+ 72us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.452037 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.452174 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260927 18:38:07.452409 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.445668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.6.4.62:43091)
0927 18:38:07.445786 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.445790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.445867 (+ 77us) server_negotiation.cc:407] Connection header received
0927 18:38:07.445957 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.445960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.446004 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.446095 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.446450 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.447252 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.448010 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.448377 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.448465 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.449445 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.449449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.449452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.449475 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.450917 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.451292 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.451296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.451297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.451389 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.451598 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.451601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.451602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.451807 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.451902 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.452138 (+ 236us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.452252 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:38:07.453073 12276 tablet_copy_service.cc:140] P 18fff560805b45478e311f545c5fbede: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9eacdc885dfb4f8d94656268bf781c60 ({username='oryx', principal='oryx/127.6.4.61@KRBTEST.COM'} at 127.0.0.1:56394)
I20260927 18:38:07.453245 12276 tablet_copy_service.cc:161] P 18fff560805b45478e311f545c5fbede: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9eacdc885dfb4f8d94656268bf781c60 at {username='oryx', principal='oryx/127.6.4.61@KRBTEST.COM'} at 127.0.0.1:56394: session id = 9eacdc885dfb4f8d94656268bf781c60-00000000000000000000000000000000
I20260927 18:38:07.454963 12276 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:38:07.456485 12720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 18:38:07.458372 12720 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: tablet copy: Starting download of 0 data blocks...
I20260927 18:38:07.458530 12720 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: tablet copy: Starting download of 1 WAL segments...
I20260927 18:38:07.459941 12720 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:38:07.461362 12276 tablet_copy_service.cc:342] P 18fff560805b45478e311f545c5fbede: Request end of tablet copy session 9eacdc885dfb4f8d94656268bf781c60-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.4.61@KRBTEST.COM'} at 127.0.0.1:56394
I20260927 18:38:07.461455 12276 tablet_copy_service.cc:433] P 18fff560805b45478e311f545c5fbede: ending tablet copy session 9eacdc885dfb4f8d94656268bf781c60-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9eacdc885dfb4f8d94656268bf781c60
I20260927 18:38:07.467782 12720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:07.469372 12829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.469406 12826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.469445 12720 server_base.cc:1034] running on GCE node
W20260927 18:38:07.469583 12827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.469781 12720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:07.470032 12720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:07.471165 12720 hybrid_clock.cc:648] HybridClock initialized: now 1790534287471152 us; error 30 us; skew 500 ppm
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:07.472579 12720 init.cc:377] Logged in from keytab as oryx/127.6.4.61@KRBTEST.COM (short username oryx)
I20260927 18:38:07.472988 12720 webserver.cc:460] Webserver started at http://127.6.4.61:39385/ using document root <none> and password file <none>
I20260927 18:38:07.473158 12720 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.473206 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.474344 12720 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:07.475185 12836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.475404 12720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:07.475471 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
uuid: "9eacdc885dfb4f8d94656268bf781c60"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7b995da6eb534add0f376d9a0193a318"
server_key_iv: "9401d9c40afbe068c9a21109b11c5529"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.475545 12720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.485684 12720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:07.487336 12720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:07.491778 12720 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.61:42643
I20260927 18:38:07.491809 12888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.61:42643 every 8 connection(s)
I20260927 18:38:07.492460 12720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
I20260927 18:38:07.493854 12889 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:38:07.494606 12889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Bootstrap starting.
I20260927 18:38:07.502419 12889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:38:07.502660 12889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: Bootstrap complete.
I20260927 18:38:07.503084 12889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } }
I20260927 18:38:07.503188 12889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9eacdc885dfb4f8d94656268bf781c60, State: Initialized, Role: LEARNER
I20260927 18:38:07.503281 12889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } }
I20260927 18:38:07.503556 12892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } }
I20260927 18:38:07.503688 12889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:07.503724 12892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: LEARNER
I20260927 18:38:07.506071 12903 catalog_manager.cc:1259] Loaded cluster ID: 5cc6d271cd784999a345512aad417549
I20260927 18:38:07.506117 12903 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: loading cluster ID for follower catalog manager: success
I20260927 18:38:07.507860 12903 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: acquiring CA information for follower catalog manager: success
I20260927 18:38:07.509614 12903 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:38:07.534328 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.526937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.6.4.61:42643)
0927 18:38:07.527217 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.527222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.527235 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:07.527277 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.527280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.527324 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.527398 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.528063 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.528817 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.529621 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.529910 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.530012 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.531372 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.531376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.531378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.531405 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.532794 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.533187 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.533191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.533193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.533267 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.533509 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.533512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.533514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.533704 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.533820 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.534053 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.534163 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.6.4.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.60.keytab.
Entry for principal oryx/127.6.4.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.60.keytab.
I20260927 18:38:07.559831 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:34305
--webserver_interface=127.6.4.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.60.keytab
--principal=oryx/127.6.4.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:07.667148 12907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:07.667395 12907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:07.667433 12907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:07.671235 12907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:07.671280 12907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:07.671298 12907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:07.671312 12907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:07.671325 12907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:07.676369 12907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.60.keytab
--principal=oryx/127.6.4.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.4.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:07.677269 12907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:07.678144 12907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:07.685966 12915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.686007 12913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.685946 12912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.686223 12907 server_base.cc:1034] running on GCE node
I20260927 18:38:07.686709 12907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:07.687708 12907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:07.688864 12907 hybrid_clock.cc:648] HybridClock initialized: now 1790534287688847 us; error 28 us; skew 500 ppm
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:07.691344 12907 init.cc:377] Logged in from keytab as oryx/127.6.4.60@KRBTEST.COM (short username oryx)
I20260927 18:38:07.692169 12907 webserver.cc:460] Webserver started at http://127.6.4.60:44779/ using document root <none> and password file <none>
I20260927 18:38:07.692660 12907 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.692720 12907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.692888 12907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:07.694765 12907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/instance:
uuid: "b2ce58595eff452bbeda36782b6cd1da"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7f33403536dac5fafeb738dc3d560415"
server_key_iv: "79b3ff65b082dc02ff32dfc0551fe3f9"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.695241 12907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal/instance:
uuid: "b2ce58595eff452bbeda36782b6cd1da"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7f33403536dac5fafeb738dc3d560415"
server_key_iv: "79b3ff65b082dc02ff32dfc0551fe3f9"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.698580 12907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:38:07.700770 12922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.701859 12907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:07.701975 12907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
uuid: "b2ce58595eff452bbeda36782b6cd1da"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7f33403536dac5fafeb738dc3d560415"
server_key_iv: "79b3ff65b082dc02ff32dfc0551fe3f9"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.702060 12907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.716089 12907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:07.718597 12907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:07.718746 12907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:07.725346 12907 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.60:34305
I20260927 18:38:07.725590 12974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.60:34305 every 8 connection(s)
I20260927 18:38:07.726244 12907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
I20260927 18:38:07.728868 12975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:07.732402 12975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:38:07.735158 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 12907
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.60@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.4.60@KRBTEST.COM
I20260927 18:38:07.745167 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.734177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56404 (local address 127.6.4.62:43091)
0927 18:38:07.734336 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.734341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.735037 (+ 696us) server_negotiation.cc:407] Connection header received
0927 18:38:07.736078 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.736082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.736156 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.736250 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.737266 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.737774 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.738950 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.739156 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.739247 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.741750 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.741754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.741757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.741784 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.743216 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.743706 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.743709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.743711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.743793 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.744131 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.744134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.744135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.744360 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.744432 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.744906 (+ 474us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.745010 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 18:38:07.745864 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.734452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56404)
0927 18:38:07.734883 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.734921 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.735815 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.736403 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.736410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.736715 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.737129 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.737141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.737891 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.737894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.738843 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.738852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.739235 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.739282 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.739440 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.740007 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.740021 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.741611 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.743319 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.743326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.743333 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.743620 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.743871 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.743874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.743875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.744028 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.744449 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.744455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.744663 (+ 208us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.744851 (+ 188us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.745121 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":35,"spinlock_wait_cycles":2816,"thread_start_us":100,"threads_started":1}
I20260927 18:38:07.746313 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.735905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.6.4.60:34305)
0927 18:38:07.736288 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.736297 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.736314 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:38:07.736369 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.736373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.736523 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.736667 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.737368 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.738208 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.738966 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.739316 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.739463 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.742163 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.742168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.742177 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.742199 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.743703 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.744151 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.744156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.744158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.744229 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.744464 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.744467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.744469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.744677 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.744823 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.745049 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.745166 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":314,"spinlock_wait_cycles":768,"thread_start_us":130,"threads_started":1}
I20260927 18:38:07.747078 12975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.60@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
I20260927 18:38:07.754103 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.747486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48750)
0927 18:38:07.747611 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.747620 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.747690 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.747897 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.747899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.748040 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.748159 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.748166 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.748861 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.748863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.749362 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.749370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.749871 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.749882 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.749948 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.750323 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.750333 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.751561 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.753035 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.753038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.753039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.753212 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.753400 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.753401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.753403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.753461 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.753772 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.753775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.753845 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.753921 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.754004 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260927 18:38:07.754251 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.747599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.6.4.61:42643)
0927 18:38:07.747723 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.747727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.747740 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:07.747778 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.747781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.747815 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.747877 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.748254 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.748766 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.749463 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.749789 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.749887 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.751677 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.751680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.751682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.751704 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.752945 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.753280 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.753282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.753283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.753344 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.753515 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.753517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.753518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.753678 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.753775 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.754016 (+ 241us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.754115 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:38:07.754797 12975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.60@KRBTEST.COM for oryx/127.6.4.60@KRBTEST.COM
I20260927 18:38:07.761466 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.755127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40914)
0927 18:38:07.755224 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.755237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.755298 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.755444 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.755446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.755553 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.755611 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.755615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.756285 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.756287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.756773 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.756781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.757551 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.757563 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.757633 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.758018 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.758028 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.759065 (+ 1037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.760357 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.760359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.760360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.760523 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.760739 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.760742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.760743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.760821 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.761097 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.761099 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.761169 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.761245 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.761337 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260927 18:38:07.761492 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.755234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.6.4.60:34305)
0927 18:38:07.755302 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.755305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.755314 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:38:07.755344 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.755345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.755374 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.755428 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.755682 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.756172 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.756859 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.757481 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.757560 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.759175 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.759178 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.759180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.759198 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.760273 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.760590 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.760593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.760594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.760649 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.760891 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.760893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.760894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.761023 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.761084 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.761279 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.761384 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":20}
I20260927 18:38:07.763882 12975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Bootstrap starting.
I20260927 18:38:07.765967 12975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:07.766590 12975 log.cc:824] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:07.768297 12975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: No bootstrap required, opened a new log
I20260927 18:38:07.769927 12447 heartbeater.cc:503] Master 127.6.4.62:43091 was elected leader, sending a full tablet report...
I20260927 18:38:07.771179 12975 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } }
I20260927 18:38:07.771364 12975 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:07.771409 12975 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2ce58595eff452bbeda36782b6cd1da, State: Initialized, Role: FOLLOWER
I20260927 18:38:07.771780 12975 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } }
I20260927 18:38:07.772168 12980 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } } }
I20260927 18:38:07.772276 12980 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:07.772800 12975 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: configured and running, proceeding with master startup.
W20260927 18:38:07.777076 12991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:38:07.777139 12991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:38:07.780158 12907 master_runner.cc:428] Detected that this master b2ce58595eff452bbeda36782b6cd1da is joining an existing cluster
I20260927 18:38:07.780212 12907 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.4.60:34305
I20260927 18:38:07.790799 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.782336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48756)
0927 18:38:07.783071 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.783083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.783158 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.783476 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.783478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.783840 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.783945 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.783951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.784967 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.784970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.786313 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.786323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.786958 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786971 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.787128 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.787581 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.787593 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.787911 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.789555 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.789557 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.789559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.789741 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.789978 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.789980 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.789981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.790044 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.790380 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.790383 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.790464 (+ 81us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.790538 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.790627 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":109,"thread_start_us":68,"threads_started":1}
I20260927 18:38:07.790856 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.782405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.6.4.61:42643)
0927 18:38:07.782512 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.782516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.783139 (+ 623us) server_negotiation.cc:407] Connection header received
0927 18:38:07.783250 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.783253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.783302 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.783368 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.784064 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.784864 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786434 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.786760 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786823 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.788036 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.788040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.788042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.788069 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.789464 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.789818 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.789821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.789823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.789905 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.790108 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.790111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.790113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.790305 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.790397 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.790596 (+ 199us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.790700 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:38:07.799322 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12318
I20260927 18:38:07.806057 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:34467
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:43091
--webserver_port=41833
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:38:07.809160 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.783036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.6.4.62:43091)
0927 18:38:07.783148 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.783152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.783165 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:07.783203 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.783206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.783250 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.783328 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.783951 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.784801 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786432 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.786777 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786848 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.805390 (+ 18542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.805394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.805397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.805423 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.807241 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.807821 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.807825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.807826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.807912 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.808223 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.808226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.808228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.808451 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.808517 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.808887 (+ 370us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.809006 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:07.809244 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.782760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.6.4.60:34305)
0927 18:38:07.782858 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.782862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.783419 (+ 557us) server_negotiation.cc:407] Connection header received
0927 18:38:07.783548 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.783550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.783583 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.783623 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.784133 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.784854 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786659 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.786923 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786999 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.805357 (+ 18358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.805360 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.805362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.805408 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.807768 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.808171 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.808173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.808175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.808252 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.808496 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.808498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.808500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.808700 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.808763 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.809025 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.809112 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":37}
I20260927 18:38:07.809115 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.782289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56406)
0927 18:38:07.782932 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.782945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.783030 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.783445 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.783449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.783681 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.783818 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.783825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.784938 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.784941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.786302 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.786312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.786982 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.786997 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.787116 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.804689 (+ 17573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.804702 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.805224 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.807377 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.807380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.807382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.807701 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.808020 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.808023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.808025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.808130 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.808579 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.808582 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.808695 (+ 113us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.808798 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.808934 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":70,"thread_start_us":106,"threads_started":1}
I20260927 18:38:07.810024 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.782713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40916)
0927 18:38:07.783356 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.783367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.783471 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.783729 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.783732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.783937 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.784020 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.784026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.784967 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.784970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.786570 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.786577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.787120 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.787132 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.787234 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.804583 (+ 17349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.804599 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.805248 (+ 649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.807854 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.807856 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.807858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.808061 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.808337 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.808339 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.808341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.808417 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.808816 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.808818 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.808885 (+ 67us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.809134 (+ 249us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.809871 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":28,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534286, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.62@KRBTEST.COM for oryx/127.6.4.60@KRBTEST.COM
I20260927 18:38:07.819892 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.812922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.6.4.60:34305)
0927 18:38:07.813213 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.813217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.813228 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:38:07.813287 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.813289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.813333 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.813449 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.813835 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.814413 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.815280 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.815508 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.815592 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.817356 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.817359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.817362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.817384 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.818556 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.818917 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.818920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.818921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.818987 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.819221 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.819224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.819225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.819366 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.819444 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.819673 (+ 229us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.819758 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260927 18:38:07.819900 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.812874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40922)
0927 18:38:07.813024 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.813037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.813095 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.813479 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.813482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.813656 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.813737 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.813743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.814543 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.814546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.815174 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.815183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.815593 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.815608 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.815695 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.816063 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.816076 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.817241 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.818651 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.818654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.818656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.818836 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.819063 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.819066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.819068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.819142 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.819450 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.819453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.819536 (+ 83us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.819617 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.819757 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260927 18:38:07.820994 12256 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.4.60:34305: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da"
member_type: NON_VOTER
last_known_addr {
host: "127.6.4.60"
port: 34305
}
attrs {
promote: true
}
}
dest_uuid: "18fff560805b45478e311f545c5fbede"
cas_config_opid_index: 5
I20260927 18:38:07.821512 12256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } }
I20260927 18:38:07.822496 12815 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b2ce58595eff452bbeda36782b6cd1da (127.6.4.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.823724 12815 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.4.60:34305
I20260927 18:38:07.823877 12815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.823971 12815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:07.824420 12907 master.cc:537] Master@127.6.4.60:34305 shutting down...
I20260927 18:38:07.824741 12949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:07.826225 12949 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Refusing update from remote peer 18fff560805b45478e311f545c5fbede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 18:38:07.827136 12814 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:38:07.828876 12240 consensus_peers.cc:487] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede -> Peer b2ce58595eff452bbeda36782b6cd1da (127.6.4.60:34305): Couldn't send request to peer b2ce58595eff452bbeda36782b6cd1da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:38:07.830848 12907 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:07.831193 12907 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:07.831357 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.822821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.6.4.61:42643)
0927 18:38:07.822928 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.822933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.826119 (+ 3186us) server_negotiation.cc:407] Connection header received
0927 18:38:07.826308 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.826312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.826372 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.826438 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:07.826982 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.827674 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.829538 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.830161 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.830228 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.830363 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.831144 (+ 781us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.831234 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:38:07.831382 12907 tablet_replica.cc:330] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: stopping tablet replica
I20260927 18:38:07.831732 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.822598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48768)
0927 18:38:07.826048 (+ 3450us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.826062 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.826182 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.826757 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.826760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.826779 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:07.826856 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.826861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.827820 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.827823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.828980 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.828991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.830891 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.830894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.830984 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.831030 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.831536 (+ 506us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.831611 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3381}
I20260927 18:38:07.832437 12863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 LEARNER]: Refusing update from remote peer 18fff560805b45478e311f545c5fbede: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 18:38:07.832854 12814 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 18:38:07.834844 12863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b2ce58595eff452bbeda36782b6cd1da (127.6.4.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.837324 12892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18fff560805b45478e311f545c5fbede. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.837417 12892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: LEARNER
I20260927 18:38:07.837503 12892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: NON_VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: true } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.837574 12892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: LEARNER
I20260927 18:38:07.838143 12814 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: attempting to promote NON_VOTER 9eacdc885dfb4f8d94656268bf781c60 to VOTER
I20260927 18:38:07.838521 12814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } }
I20260927 18:38:07.839638 12863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 LEARNER]: Refusing update from remote peer 18fff560805b45478e311f545c5fbede: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 18:38:07.839996 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.839449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40934)
0927 18:38:07.839861 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.839931 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345}
I20260927 18:38:07.839998 13003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 18:38:07.840834 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.840469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40944)
0927 18:38:07.840697 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.840764 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:34305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
W20260927 18:38:07.840929 12240 consensus_peers.cc:487] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede -> Peer b2ce58595eff452bbeda36782b6cd1da (127.6.4.60:34305): Couldn't send request to peer b2ce58595eff452bbeda36782b6cd1da. Status: Network error: Client connection negotiation failed: client connection to 127.6.4.60:34305: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 18:38:07.841732 12815 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.842198 12863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.843118 12814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.843237 12814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:07.843153 12815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.843405 12815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:07.843927 12892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.844024 12892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:07.847250 12907 master.cc:552] Master@127.6.4.60:34305 shutdown complete.
I20260927 18:38:07.847291 12907 master_runner.cc:305] Clearing existing system tablet
I20260927 18:38:07.847482 12907 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.847524 12907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.848551 12907 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:38:07.849259 13007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.849424 12907 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.849547 12907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:07.849583 12907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
uuid: "b2ce58595eff452bbeda36782b6cd1da"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7f33403536dac5fafeb738dc3d560415"
server_key_iv: "79b3ff65b082dc02ff32dfc0551fe3f9"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.851068 12907 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:07.852836 12907 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 18:38:07.852911 12907 log.cc:1196] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 18:38:07.853106 12907 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Deleting consensus metadata
I20260927 18:38:07.853333 12907 master_runner.cc:315] Copying system tablet from 127.6.4.62:43091
I20260927 18:38:07.854257 12907 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: tablet copy: Beginning tablet copy session from remote peer at address 127.6.4.62:43091
I20260927 18:38:07.861088 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.854821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56412 (local address 127.6.4.62:43091)
0927 18:38:07.854917 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:07.854921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:07.855131 (+ 210us) server_negotiation.cc:407] Connection header received
0927 18:38:07.855229 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:07.855232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:07.855274 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:07.855337 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:07.855762 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.856339 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.857125 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.857341 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.857421 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.858389 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:07.858392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:07.858395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:07.858420 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:07.859634 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.860002 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.860005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.860007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.860074 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:07.860380 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:07.860384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:07.860385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:07.860547 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:07.860629 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:07.860856 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 18:38:07.860950 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:38:07.861102 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:07.854772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56412)
0927 18:38:07.855053 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:07.855063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:07.855157 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:07.855361 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:07.855363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:07.855541 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:07.855630 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.855635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.856433 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.856437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.857025 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:07.857033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:07.857420 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:07.857436 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:07.857523 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:07.857969 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:07.857982 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:07.858272 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:07.859736 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.859739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.859741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.859920 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.860203 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:07.860206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:07.860209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:07.860297 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:07.860630 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:07.860633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:07.860717 (+ 84us) client_negotiation.cc:769] Sending connection context
0927 18:38:07.860797 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:38:07.860925 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
I20260927 18:38:07.861719 12276 tablet_copy_service.cc:140] P 18fff560805b45478e311f545c5fbede: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b2ce58595eff452bbeda36782b6cd1da ({username='oryx', principal='oryx/127.6.4.60@KRBTEST.COM'} at 127.0.0.1:56412)
I20260927 18:38:07.861793 12276 tablet_copy_service.cc:161] P 18fff560805b45478e311f545c5fbede: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b2ce58595eff452bbeda36782b6cd1da at {username='oryx', principal='oryx/127.6.4.60@KRBTEST.COM'} at 127.0.0.1:56412: session id = b2ce58595eff452bbeda36782b6cd1da-00000000000000000000000000000000
I20260927 18:38:07.862516 12276 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:38:07.863466 12907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 18:38:07.865135 12907 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: tablet copy: Starting download of 0 data blocks...
I20260927 18:38:07.865295 12907 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: tablet copy: Starting download of 1 WAL segments...
I20260927 18:38:07.866456 12907 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:38:07.867749 12276 tablet_copy_service.cc:342] P 18fff560805b45478e311f545c5fbede: Request end of tablet copy session b2ce58595eff452bbeda36782b6cd1da-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.4.60@KRBTEST.COM'} at 127.0.0.1:56412
I20260927 18:38:07.867815 12276 tablet_copy_service.cc:433] P 18fff560805b45478e311f545c5fbede: ending tablet copy session b2ce58595eff452bbeda36782b6cd1da-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b2ce58595eff452bbeda36782b6cd1da
I20260927 18:38:07.873667 12907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:07.875073 13014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.875200 13017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.875391 12907 server_base.cc:1034] running on GCE node
W20260927 18:38:07.875571 13015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.875765 12907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:07.876022 12907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:07.877153 12907 hybrid_clock.cc:648] HybridClock initialized: now 1790534287877143 us; error 24 us; skew 500 ppm
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:07.878400 12907 init.cc:377] Logged in from keytab as oryx/127.6.4.60@KRBTEST.COM (short username oryx)
I20260927 18:38:07.878818 12907 webserver.cc:460] Webserver started at http://127.6.4.60:40865/ using document root <none> and password file <none>
I20260927 18:38:07.878988 12907 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.879025 12907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.880012 12907 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:07.880856 13024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.881081 12907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:07.881142 12907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
uuid: "b2ce58595eff452bbeda36782b6cd1da"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "7f33403536dac5fafeb738dc3d560415"
server_key_iv: "79b3ff65b082dc02ff32dfc0551fe3f9"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.881219 12907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:07.912634 12907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:07.914430 12907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:07.918921 12907 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.60:34305
I20260927 18:38:07.918937 13076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.60:34305 every 8 connection(s)
I20260927 18:38:07.919405 12907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
I20260927 18:38:07.920753 13077 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:38:07.921519 13077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Bootstrap starting.
I20260927 18:38:07.929821 13077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:38:07.930066 13077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Bootstrap complete.
I20260927 18:38:07.930541 13077 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } }
I20260927 18:38:07.930665 13077 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b2ce58595eff452bbeda36782b6cd1da, State: Initialized, Role: LEARNER
I20260927 18:38:07.930769 13077 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } }
I20260927 18:38:07.931046 13080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:07.931180 13077 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:07.931208 13080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: This master's current role is: LEARNER
I20260927 18:38:07.934468 13091 catalog_manager.cc:1259] Loaded cluster ID: 5cc6d271cd784999a345512aad417549
I20260927 18:38:07.934512 13091 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: loading cluster ID for follower catalog manager: success
I20260927 18:38:07.935947 13091 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: acquiring CA information for follower catalog manager: success
I20260927 18:38:07.937022 13091 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 18:38:07.959582 13000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:07.959893 13000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:07.959952 13000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:07.965523 13000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:07.965600 13000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:07.965698 13000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:07.967597 12581 heartbeater.cc:503] Master 127.6.4.62:43091 was elected leader, sending a full tablet report...
I20260927 18:38:07.971737 13000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:34467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=41833
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:07.972775 13000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:07.973867 13000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:07.983560 13097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.983556 13096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:07.983556 13099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:07.983747 13000 server_base.cc:1034] running on GCE node
I20260927 18:38:07.984131 13000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:07.984764 13000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:07.985934 13000 hybrid_clock.cc:648] HybridClock initialized: now 1790534287985907 us; error 38 us; skew 500 ppm
Sep 27 18:38:07 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:07.988682 13000 init.cc:377] Logged in from keytab as oryx/127.6.4.1@KRBTEST.COM (short username oryx)
I20260927 18:38:07.989455 13000 webserver.cc:460] Webserver started at http://127.6.4.1:41833/ using document root <none> and password file <none>
I20260927 18:38:07.989959 13000 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:07.990034 13000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:07.994033 13000 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 18:38:07.996147 13106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:07.997470 13000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:07.997568 13000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "d8079d590210424dbdeac88bbe706045"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "14410e20a0cfcd2847c1157535674755"
server_key_iv: "1d869f14e4ad0a7d755e2be8d30dd965"
server_key_version: "encryptionkey@0"
I20260927 18:38:07.998086 13000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:08.015908 13000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:08.018536 13000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:08.018705 13000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:08.019236 13000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:08.020030 13000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:08.020076 13000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.020156 13000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:08.020181 13000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.056505 13000 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:34467
I20260927 18:38:08.056552 13219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:34467 every 8 connection(s)
I20260927 18:38:08.057595 13000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:08.064361 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 13000
I20260927 18:38:08.064544 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12452
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.60@KRBTEST.COM
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534287, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:08.071542 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:33529
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:43091
--webserver_port=33543
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:38:08.074517 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.059477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:55909 (local address 127.6.4.62:43091)
0927 18:38:08.059619 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.059623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.060740 (+ 1117us) server_negotiation.cc:407] Connection header received
0927 18:38:08.061742 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.061746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.061797 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.061847 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.063223 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.063999 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.067245 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.067589 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.067680 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.070213 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.070217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.070220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.070249 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.072360 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072977 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.072980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.072982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.073059 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.073339 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.073342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.073344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.073540 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.073626 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.074244 (+ 618us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.074356 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:38:08.074582 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.059941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:42639 (local address 127.6.4.60:34305)
0927 18:38:08.060189 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.060194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.061103 (+ 909us) server_negotiation.cc:407] Connection header received
0927 18:38:08.061716 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.061719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.061774 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.061853 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.063184 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.064237 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.065610 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.065960 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.066032 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.070451 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.070459 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.070463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.070491 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.072198 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072679 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.072683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.072685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.072752 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.073181 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.073184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.073186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.073390 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.073455 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.074282 (+ 827us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.074415 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
I20260927 18:38:08.075132 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.059477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44579 (local address 127.6.4.61:42643)
0927 18:38:08.059610 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.059615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.061160 (+ 1545us) server_negotiation.cc:407] Connection header received
0927 18:38:08.061741 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.061745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.061797 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.061847 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.064179 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.064861 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.066003 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.066212 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.066298 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.069936 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.069939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.069942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.069967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.071446 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072649 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.072653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.072655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.072745 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.073165 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.073168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.073169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.073398 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.073478 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.074885 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.074991 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:38:08.075555 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.060019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.6.4.1:42639)
0927 18:38:08.061010 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.061022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.061528 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.062030 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.062037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.062483 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.063051 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.063064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.064353 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.064356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.065483 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.065493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.066166 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.066217 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.066425 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.067832 (+ 1407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.067846 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.070053 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.072320 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.072325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.072581 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.072850 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.072855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.073065 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.073527 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.073533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.074073 (+ 540us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.074224 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.074676 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":274,"spinlock_wait_cycles":48512,"thread_start_us":97,"threads_started":1}
I20260927 18:38:08.075827 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.060019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.6.4.1:44579)
0927 18:38:08.061063 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.061075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.061535 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.062032 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.062039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.062489 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.063065 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.063074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.064978 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.064981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.065848 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.065858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.066296 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.066311 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.066418 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.067140 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.067159 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.069754 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.072163 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.072184 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.072549 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.072848 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.072852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.073059 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.073526 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.073532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.073964 (+ 432us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.074162 (+ 198us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.074521 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":307,"spinlock_wait_cycles":26368,"thread_start_us":77,"threads_started":1}
I20260927 18:38:08.076882 13222 heartbeater.cc:347] Connected to a master server at 127.6.4.61:42643
I20260927 18:38:08.077107 13222 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.077548 13222 heartbeater.cc:511] Master 127.6.4.61:42643 requested a full tablet report, sending...
I20260927 18:38:08.077992 13220 heartbeater.cc:347] Connected to a master server at 127.6.4.60:34305
I20260927 18:38:08.078069 13220 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.078239 13220 heartbeater.cc:511] Master 127.6.4.60:34305 requested a full tablet report, sending...
I20260927 18:38:08.079066 12853 ts_manager.cc:194] Registered new tserver with Master: d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467)
I20260927 18:38:08.079136 13041 ts_manager.cc:194] Registered new tserver with Master: d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467)
I20260927 18:38:08.075568 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.060019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:55909)
0927 18:38:08.060537 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.060587 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.061524 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.062032 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.062039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.062486 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.063068 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.063074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.064148 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.064160 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.065874 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.065881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.067677 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.067702 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.067843 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.068682 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.068697 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.070053 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.072650 (+ 2597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.072653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.072654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.072869 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.073164 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.073167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.073169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.073245 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.073678 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.073680 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.073964 (+ 284us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.074980 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.075093 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":166,"spinlock_wait_cycles":16384,"thread_start_us":145,"threads_started":1}
I20260927 18:38:08.080482 13221 heartbeater.cc:347] Connected to a master server at 127.6.4.62:43091
I20260927 18:38:08.080577 13221 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.080727 13221 heartbeater.cc:511] Master 127.6.4.62:43091 requested a full tablet report, sending...
I20260927 18:38:08.081455 12256 ts_manager.cc:194] Re-registered known tserver with Master: d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467)
I20260927 18:38:08.082020 12256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:55909
I20260927 18:38:08.161331 12715 heartbeater.cc:503] Master 127.6.4.62:43091 was elected leader, sending a full tablet report...
W20260927 18:38:08.180820 13228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:08.181042 13228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:08.181099 13228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:08.185019 13228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:08.185066 13228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:08.185142 13228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:08.190071 13228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:33529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=33543
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:08.190989 13228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:08.191937 13228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:08.200877 13233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:08.200865 13236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:08.200866 13234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:08.201150 13228 server_base.cc:1034] running on GCE node
I20260927 18:38:08.201511 13228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:08.202093 13228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:08.203251 13228 hybrid_clock.cc:648] HybridClock initialized: now 1790534288203237 us; error 30 us; skew 500 ppm
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:08.205909 13228 init.cc:377] Logged in from keytab as oryx/127.6.4.2@KRBTEST.COM (short username oryx)
I20260927 18:38:08.206733 13228 webserver.cc:460] Webserver started at http://127.6.4.2:33543/ using document root <none> and password file <none>
I20260927 18:38:08.207226 13228 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:08.207309 13228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:08.210807 13228 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:08.212666 13243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.213829 13228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:08.213937 13228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "0a428c09443549899f21249cf969ff9c"
format_stamp: "Formatted at 2026-09-27 18:38:06 on dist-test-slave-bzzj"
server_key: "f933199d0c5f0f85a2f893c73d189909"
server_key_iv: "76b075778a104d9f1d3b5da673363ada"
server_key_version: "encryptionkey@0"
I20260927 18:38:08.214450 13228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:08.250026 13228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:08.252943 13228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:08.253084 13228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:08.253566 13228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:08.254350 13228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:08.254395 13228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.254433 13228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:08.254453 13228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.262390 13228 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:33529
I20260927 18:38:08.262444 13356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:33529 every 8 connection(s)
I20260927 18:38:08.263284 13228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:08.269275 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 13228
I20260927 18:38:08.269441 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12586
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.60@KRBTEST.COM
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:08.276746 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:34573
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:43091
--webserver_port=38027
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
I20260927 18:38:08.277667 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.265017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51965 (local address 127.6.4.60:34305)
0927 18:38:08.265143 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.265148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.266265 (+ 1117us) server_negotiation.cc:407] Connection header received
0927 18:38:08.267180 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.267183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.267232 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.267290 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.268511 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.269023 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.270472 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.270807 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.270885 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.273314 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.273318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.273321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.273349 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.275009 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.275540 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.275544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.275546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.275628 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.276381 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.276384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.276386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.276585 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.276652 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.277402 (+ 750us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.277519 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:38:08.277683 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.264942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:56161 (local address 127.6.4.62:43091)
0927 18:38:08.265063 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.265067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.266057 (+ 990us) server_negotiation.cc:407] Connection header received
0927 18:38:08.267158 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.267162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.267217 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.267289 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.268490 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.269290 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.270424 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.270744 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.270838 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.273750 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.273753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.273756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.273784 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.275607 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.276047 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.276051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.276053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.276151 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.276467 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.276470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.276472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.276688 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.276757 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.277408 (+ 651us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.277517 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:38:08.279109 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.265314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.6.4.2:51965)
0927 18:38:08.266173 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.266185 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.266969 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.267466 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.267473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.267873 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.268376 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.268385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.269182 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.269184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.270371 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.270381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.270975 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.270991 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.271097 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.271582 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.271603 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.273132 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.275125 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.275130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.275143 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.275437 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.276080 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.276083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.276085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.276254 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.276726 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.276732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.277158 (+ 426us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.278168 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.278309 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":131,"spinlock_wait_cycles":14592,"thread_start_us":124,"threads_started":1}
I20260927 18:38:08.279119 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.265314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:56161)
0927 18:38:08.265927 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.265973 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.266974 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.267464 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.267470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.267828 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.268365 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.268379 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.269419 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.269422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.270316 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.270324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.270832 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.270876 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.271097 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.271717 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.271730 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.273542 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.275748 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.275751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.275753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.275963 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.276266 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.276269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.276271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.276358 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.276895 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.276898 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.277151 (+ 253us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.277826 (+ 675us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.278126 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":93,"spinlock_wait_cycles":6528,"thread_start_us":124,"threads_started":1}
I20260927 18:38:08.280591 13358 heartbeater.cc:347] Connected to a master server at 127.6.4.62:43091
I20260927 18:38:08.280599 13357 heartbeater.cc:347] Connected to a master server at 127.6.4.60:34305
I20260927 18:38:08.280822 13357 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.280821 13358 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.281383 13357 heartbeater.cc:511] Master 127.6.4.60:34305 requested a full tablet report, sending...
I20260927 18:38:08.281448 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.265603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:38805 (local address 127.6.4.61:42643)
0927 18:38:08.265710 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.265714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.266319 (+ 605us) server_negotiation.cc:407] Connection header received
0927 18:38:08.267180 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.267183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.267232 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.267282 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.268518 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.269306 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.274887 (+ 5581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.275260 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.275340 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.277524 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.277527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.277530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.277554 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.279169 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.279636 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.279639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.279640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.279741 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.280257 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.280261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.280262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.280475 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.280539 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.281198 (+ 659us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.281309 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:38:08.281383 13358 heartbeater.cc:511] Master 127.6.4.62:43091 requested a full tablet report, sending...
I20260927 18:38:08.281878 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.265491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.6.4.2:38805)
0927 18:38:08.266247 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.266264 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.266969 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.267466 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.267473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.267874 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.268378 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.268385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.273880 (+ 5495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.273884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.274756 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.274767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.275477 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.275546 (+ 69us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.275645 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.276005 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.276021 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.277399 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.279311 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.279313 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.279315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.279526 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.280041 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.280044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.280046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.280150 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.280871 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.280873 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.280962 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.281604 (+ 642us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.281729 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":102,"spinlock_wait_cycles":19840,"thread_start_us":97,"threads_started":1}
I20260927 18:38:08.282362 12256 ts_manager.cc:194] Re-registered known tserver with Master: 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:08.282465 13359 heartbeater.cc:347] Connected to a master server at 127.6.4.61:42643
I20260927 18:38:08.282531 13359 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.282519 13041 ts_manager.cc:194] Registered new tserver with Master: 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:08.282667 13359 heartbeater.cc:511] Master 127.6.4.61:42643 requested a full tablet report, sending...
I20260927 18:38:08.283043 12256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:56161
I20260927 18:38:08.283378 12853 ts_manager.cc:194] Registered new tserver with Master: 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
W20260927 18:38:08.382248 13364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:08.382457 13364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:08.382500 13364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:08.386360 13364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:08.386405 13364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:08.386469 13364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:08.391371 13364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:34573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=38027
--tserver_master_addrs=127.6.4.62:43091,127.6.4.61:42643,127.6.4.60:34305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:08.392285 13364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:08.393177 13364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:08.401784 13370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:08.401852 13369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:08.401896 13372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:08.401954 13364 server_base.cc:1034] running on GCE node
I20260927 18:38:08.402298 13364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:08.402848 13364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:08.403990 13364 hybrid_clock.cc:648] HybridClock initialized: now 1790534288403970 us; error 29 us; skew 500 ppm
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:08.406482 13364 init.cc:377] Logged in from keytab as oryx/127.6.4.3@KRBTEST.COM (short username oryx)
I20260927 18:38:08.407251 13364 webserver.cc:460] Webserver started at http://127.6.4.3:38027/ using document root <none> and password file <none>
I20260927 18:38:08.407722 13364 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:08.407801 13364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:08.411460 13364 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:08.413636 13380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.414135 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.409546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.6.4.60:34305)
0927 18:38:08.409649 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.409653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.410641 (+ 988us) server_negotiation.cc:407] Connection header received
0927 18:38:08.410783 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.410786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.410855 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.410941 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:08.411200 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.411743 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.413120 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.413648 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.413766 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.413907 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.413978 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.414038 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:38:08.414165 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.409495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40946)
0927 18:38:08.410556 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.410567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.410660 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.410976 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.410979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.411000 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:08.411105 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.411112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.411851 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.411855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.413002 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.413013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.413794 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.413798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.413865 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.413887 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.413949 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.414026 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":60,"threads_started":1}
I20260927 18:38:08.414677 13043 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260927 18:38:08.414939 13364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:08.415004 13080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18fff560805b45478e311f545c5fbede. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:08.415134 13080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: This master's current role is: LEARNER
I20260927 18:38:08.415035 13364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "41d9d386cbd44064877578ce2fe3c0e0"
format_stamp: "Formatted at 2026-09-27 18:38:07 on dist-test-slave-bzzj"
server_key: "794da5e4a98b13d8f0a96c7101ecf212"
server_key_iv: "8f4c10bd5021a8f8d67a5fa805d0c423"
server_key_version: "encryptionkey@0"
I20260927 18:38:08.415587 13364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:08.431303 13364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:08.434185 13364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:08.434331 13364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:08.434823 13364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:08.435554 13364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:08.435597 13364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.435633 13364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:08.435649 13364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:08.443183 13364 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:34573
I20260927 18:38:08.443230 13493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:34573 every 8 connection(s)
I20260927 18:38:08.444021 13364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:08.453084 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 13364
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.60@KRBTEST.COM
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
Sep 27 18:38:08 dist-test-slave-bzzj krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534288, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.61@KRBTEST.COM
I20260927 18:38:08.459283 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.445634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:41341 (local address 127.6.4.61:42643)
0927 18:38:08.445752 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.445757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.446998 (+ 1241us) server_negotiation.cc:407] Connection header received
0927 18:38:08.447795 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.447798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.447841 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.447909 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.449181 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.449972 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451682 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.451988 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.452073 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.455121 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.455124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.455127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.455151 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.457317 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457794 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.457797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.457799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.457880 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.458135 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.458138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.458139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.458365 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.458448 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.459014 (+ 566us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.459119 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:38:08.459362 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.445620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:46921 (local address 127.6.4.60:34305)
0927 18:38:08.445740 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.445744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.446807 (+ 1063us) server_negotiation.cc:407] Connection header received
0927 18:38:08.448184 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.448187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.448242 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.448323 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.449217 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.449971 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451216 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.451422 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451502 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.455422 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.455426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.455429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.455454 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.457233 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457789 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.457792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.457794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.458001 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.458548 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.458551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.458553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.458736 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.458817 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.459136 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.459234 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:38:08.459960 13498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.446030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.6.4.3:46921)
0927 18:38:08.446675 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.446723 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.448082 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.448344 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.448347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.448613 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.449054 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.449068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.450075 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.450078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.451117 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.451125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.451499 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451542 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.451714 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.452269 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.452289 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.455277 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.457360 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.457365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.457683 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.458369 (+ 686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.458372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.458374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.458453 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.458884 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.458887 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.458974 (+ 87us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.459610 (+ 636us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.459730 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":151,"spinlock_wait_cycles":27264,"thread_start_us":128,"threads_started":1}
I20260927 18:38:08.460234 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.446220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:33409)
0927 18:38:08.447900 (+ 1680us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.447909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.447982 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.448297 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.448300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.448616 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.449054 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.449068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.449859 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.449861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.451119 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.451128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.451772 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451784 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.451845 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.452384 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.452395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.454456 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.457070 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.457089 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.457410 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.457696 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.457701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.457852 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.458271 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.458277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.458550 (+ 273us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.458732 (+ 182us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.459037 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1633,"mutex_wait_us":115,"spinlock_wait_cycles":30848,"thread_start_us":89,"threads_started":1}
I20260927 18:38:08.460762 13499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.446030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.6.4.3:41341)
0927 18:38:08.446921 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.446930 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.447588 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.448082 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.448088 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.448609 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:08.449052 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.449064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.450823 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.450826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.451583 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.451591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.452073 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.452121 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.452207 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.452558 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:08.452571 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:08.454937 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:08.457411 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.457416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.457683 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.457961 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:08.457966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:08.458046 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:08.458649 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:08.458652 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:08.458729 (+ 77us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.458828 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.460593 (+ 1765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":1633,"spinlock_wait_cycles":27392,"thread_start_us":132,"threads_started":1}
I20260927 18:38:08.461023 13494 heartbeater.cc:347] Connected to a master server at 127.6.4.60:34305
I20260927 18:38:08.461199 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.446339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:33409 (local address 127.6.4.62:43091)
0927 18:38:08.446447 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.446451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.447982 (+ 1531us) server_negotiation.cc:407] Connection header received
0927 18:38:08.448053 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.448057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.448123 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.448199 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.449212 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.449727 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451253 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.451583 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.451653 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.454599 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.454603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.454606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.454633 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.456947 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457516 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.457520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.457522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.457605 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.457942 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.457945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.457947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.458132 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.458223 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.460950 (+ 2727us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.461055 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:08.461239 13494 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.461385 13496 heartbeater.cc:347] Connected to a master server at 127.6.4.61:42643
I20260927 18:38:08.461464 13496 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.461699 13494 heartbeater.cc:511] Master 127.6.4.60:34305 requested a full tablet report, sending...
I20260927 18:38:08.461699 13496 heartbeater.cc:511] Master 127.6.4.61:42643 requested a full tablet report, sending...
I20260927 18:38:08.462515 13495 heartbeater.cc:347] Connected to a master server at 127.6.4.62:43091
I20260927 18:38:08.462589 13495 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:08.462711 12853 ts_manager.cc:194] Registered new tserver with Master: 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573)
I20260927 18:38:08.462728 13495 heartbeater.cc:511] Master 127.6.4.62:43091 requested a full tablet report, sending...
I20260927 18:38:08.463136 13041 ts_manager.cc:194] Registered new tserver with Master: 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573)
I20260927 18:38:08.463271 12256 ts_manager.cc:194] Re-registered known tserver with Master: 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573)
I20260927 18:38:08.463936 12256 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:33409
I20260927 18:38:08.466280 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.455098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.6.4.62:43091)
0927 18:38:08.455972 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.455976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.455988 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:08.456021 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.456023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.456064 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.456166 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.457244 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.457990 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.459152 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.459500 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.459586 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.462994 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.462998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.463000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.463023 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.464738 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.465159 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.465162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.465164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.465239 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.465476 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.465479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.465481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.465666 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.465785 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.466000 (+ 215us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.466114 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":64,"threads_started":1}
I20260927 18:38:08.469061 12256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:08.472679 12256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:08.472779 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.456546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.6.4.60:34305)
0927 18:38:08.459491 (+ 2945us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.459496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.464370 (+ 4874us) server_negotiation.cc:407] Connection header received
0927 18:38:08.465544 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.465547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.465588 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.465657 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.466450 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.466914 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.467905 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.468229 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.468298 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.469739 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.469742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.469743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.469763 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.471147 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.471501 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.471504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.471506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.471579 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.471952 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.471955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.471956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.472157 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.472252 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.472540 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.472633 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2888,"thread_start_us":82,"threads_started":1}
I20260927 18:38:08.472805 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.455296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.6.4.61:42643)
0927 18:38:08.457184 (+ 1888us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.457196 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.464027 (+ 6831us) server_negotiation.cc:407] Connection header received
0927 18:38:08.464191 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.464195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.464241 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.464334 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.465670 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.466334 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.467560 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.467780 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.467862 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.469263 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.469267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.469269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.469292 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.470926 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.471321 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.471324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.471326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.471406 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.471873 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.471877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.471879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.472077 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.472201 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.472539 (+ 338us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.472650 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1820,"thread_start_us":83,"threads_started":1}
I20260927 18:38:08.475435 13005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: NON_VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: true } } }
I20260927 18:38:08.475548 13005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:08.478843 13005 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: attempting to promote NON_VOTER b2ce58595eff452bbeda36782b6cd1da to VOTER
I20260927 18:38:08.479169 13005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:08.480396 12863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Refusing update from remote peer 18fff560805b45478e311f545c5fbede: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260927 18:38:08.480742 13514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260927 18:38:08.481096 13043 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 LEARNER]: Refusing update from remote peer 18fff560805b45478e311f545c5fbede: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260927 18:38:08.481452 13514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260927 18:38:08.482316 13513 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b2ce58595eff452bbeda36782b6cd1da (127.6.4.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.482715 12863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b2ce58595eff452bbeda36782b6cd1da (127.6.4.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.483704 13513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.483800 13513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:08.483906 13513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.483979 13513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:08.484066 13513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.484160 13513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:08.484014 13043 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b2ce58595eff452bbeda36782b6cd1da (127.6.4.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.485028 13522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.485247 13522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:08.485350 13080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18fff560805b45478e311f545c5fbede" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:08.485433 13080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:08.494303 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.488888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:34573 (local address 127.0.0.1:47110)
0927 18:38:08.489847 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.489859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.489937 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.490403 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.490406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.490419 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:08.490504 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.490510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.491783 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.491786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.492829 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.492864 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.493911 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.493915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.494008 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.494033 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.494103 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.494167 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":78,"threads_started":1}
I20260927 18:38:08.494367 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.489263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.6.4.3:34573)
0927 18:38:08.489678 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.489690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.490017 (+ 327us) server_negotiation.cc:407] Connection header received
0927 18:38:08.490093 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.490098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.490291 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.490431 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:08.490584 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.491664 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.492982 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.493790 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.493901 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.494036 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.494129 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.494221 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":103,"threads_started":1}
I20260927 18:38:08.494925 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.489561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.6.4.2:33529)
0927 18:38:08.490137 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.490149 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.490442 (+ 293us) server_negotiation.cc:407] Connection header received
0927 18:38:08.490514 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.490519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.490682 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.490809 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:08.490997 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.491680 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.493382 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.494171 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.494267 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.494394 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.494700 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.494793 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":85,"threads_started":1}
I20260927 18:38:08.494949 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.489207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33529 (local address 127.0.0.1:41340)
0927 18:38:08.490264 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.490276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.490359 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.490791 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.490793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.490809 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:08.490899 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.490906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.492174 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.492177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.493260 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.493270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.494456 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.494460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.494549 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.494571 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.494668 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.494751 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"thread_start_us":83,"threads_started":1}
I20260927 18:38:08.497556 13291 tablet_service.cc:1467] Processing CreateTablet for tablet 5c49cfb514c54a669d87582e7ba36834 (DEFAULT_TABLE table=test-table [id=cfeb4dfce1d649d78f2da349770bb176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:08.497560 13428 tablet_service.cc:1467] Processing CreateTablet for tablet 5c49cfb514c54a669d87582e7ba36834 (DEFAULT_TABLE table=test-table [id=cfeb4dfce1d649d78f2da349770bb176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:08.498373 13291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c49cfb514c54a669d87582e7ba36834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:08.498452 13428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c49cfb514c54a669d87582e7ba36834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:08.501725 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.488663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:34467 (local address 127.0.0.1:33040)
0927 18:38:08.488805 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.488819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.488896 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.489598 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.489601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.489617 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:08.489712 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.489719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.491069 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.491072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.492075 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.492084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.501352 (+ 9268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.501356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.501447 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.501469 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.501516 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.501609 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260927 18:38:08.502485 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.488818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.6.4.1:34467)
0927 18:38:08.489109 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.489120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.489149 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:38:08.489249 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.489254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.489477 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.489630 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:08.489833 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.490950 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.500427 (+ 9477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.501217 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.502070 (+ 853us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.502211 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.502262 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.502357 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":92,"threads_started":1}
I20260927 18:38:08.505868 13154 tablet_service.cc:1467] Processing CreateTablet for tablet 5c49cfb514c54a669d87582e7ba36834 (DEFAULT_TABLE table=test-table [id=cfeb4dfce1d649d78f2da349770bb176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:08.506954 13154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c49cfb514c54a669d87582e7ba36834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:08.510985 13528 tablet_bootstrap.cc:492] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Bootstrap starting.
I20260927 18:38:08.513105 13528 tablet_bootstrap.cc:654] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:08.513825 13528 log.cc:824] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:08.515674 13528 tablet_bootstrap.cc:492] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: No bootstrap required, opened a new log
I20260927 18:38:08.515874 13528 ts_tablet_manager.cc:1397] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:38:08.520331 13531 tablet_bootstrap.cc:492] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Bootstrap starting.
I20260927 18:38:08.520357 13530 tablet_bootstrap.cc:492] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Bootstrap starting.
I20260927 18:38:08.520853 13528 raft_consensus.cc:348] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.521095 13528 raft_consensus.cc:374] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:08.521153 13528 raft_consensus.cc:729] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41d9d386cbd44064877578ce2fe3c0e0, State: Initialized, Role: FOLLOWER
I20260927 18:38:08.521552 13528 consensus_queue.cc:260] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.522126 13495 heartbeater.cc:503] Master 127.6.4.62:43091 was elected leader, sending a full tablet report...
I20260927 18:38:08.522516 13531 tablet_bootstrap.cc:654] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:08.522809 13530 tablet_bootstrap.cc:654] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:08.524127 13528 ts_tablet_manager.cc:1428] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260927 18:38:08.524789 13531 log.cc:824] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:08.525585 13530 log.cc:824] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:08.527357 13530 tablet_bootstrap.cc:492] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: No bootstrap required, opened a new log
I20260927 18:38:08.527357 13531 tablet_bootstrap.cc:492] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: No bootstrap required, opened a new log
I20260927 18:38:08.527534 13531 ts_tablet_manager.cc:1397] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260927 18:38:08.527534 13530 ts_tablet_manager.cc:1397] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260927 18:38:08.531791 13530 raft_consensus.cc:348] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.531813 13531 raft_consensus.cc:348] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.532037 13531 raft_consensus.cc:374] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:08.532037 13530 raft_consensus.cc:374] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:08.532110 13531 raft_consensus.cc:729] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a428c09443549899f21249cf969ff9c, State: Initialized, Role: FOLLOWER
I20260927 18:38:08.532124 13530 raft_consensus.cc:729] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8079d590210424dbdeac88bbe706045, State: Initialized, Role: FOLLOWER
I20260927 18:38:08.532557 13530 consensus_queue.cc:260] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.532557 13531 consensus_queue.cc:260] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.533481 13530 ts_tablet_manager.cc:1428] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 18:38:08.533481 13531 ts_tablet_manager.cc:1428] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 18:38:08.533732 13358 heartbeater.cc:503] Master 127.6.4.62:43091 was elected leader, sending a full tablet report...
I20260927 18:38:08.534227 13221 heartbeater.cc:503] Master 127.6.4.62:43091 was elected leader, sending a full tablet report...
W20260927 18:38:08.559108 13223 tablet.cc:2369] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:08.600807 13535 raft_consensus.cc:482] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:08.600991 13535 raft_consensus.cc:504] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.602027 13535 leader_election.cc:288] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573), 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:08.606824 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.602215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:34573 (local address 127.6.4.1:42579)
0927 18:38:08.602499 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.602513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.602626 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.602881 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.602884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.602904 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:08.603076 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.603083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.604077 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.604081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.605533 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.605544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.606396 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.606400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.606503 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.606530 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.606595 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.606664 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":123,"spinlock_wait_cycles":7424,"thread_start_us":79,"threads_started":1}
I20260927 18:38:08.606827 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.602324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:42579 (local address 127.6.4.3:34573)
0927 18:38:08.602563 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.602567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.602583 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:38:08.602725 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.602727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.602780 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.602854 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:08.603179 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.603954 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.605648 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.606249 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.606352 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.606413 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.606674 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.606735 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260927 18:38:08.607334 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.602215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33529 (local address 127.6.4.1:59741)
0927 18:38:08.602514 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:08.602524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:08.602626 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:08.602886 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:08.602923 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:08.602937 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:08.603034 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.603040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.604077 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.604081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.605533 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:08.605544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.606986 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.606990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:08.607076 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.607097 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:38:08.607161 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:38:08.607228 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":104,"spinlock_wait_cycles":3584,"thread_start_us":64,"threads_started":1}
I20260927 18:38:08.607391 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.602319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:59741 (local address 127.6.4.2:33529)
0927 18:38:08.602458 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.602463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.602595 (+ 132us) server_negotiation.cc:407] Connection header received
0927 18:38:08.602748 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.602751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.602815 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.602878 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:08.603180 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.603982 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.605659 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.606249 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.606353 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.606413 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.607203 (+ 790us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.607285 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 18:38:08.607458 13448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c49cfb514c54a669d87582e7ba36834" candidate_uuid: "d8079d590210424dbdeac88bbe706045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41d9d386cbd44064877578ce2fe3c0e0" is_pre_election: true
I20260927 18:38:08.607761 13448 raft_consensus.cc:2454] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8079d590210424dbdeac88bbe706045 in term 0.
I20260927 18:38:08.607898 13311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c49cfb514c54a669d87582e7ba36834" candidate_uuid: "d8079d590210424dbdeac88bbe706045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a428c09443549899f21249cf969ff9c" is_pre_election: true
I20260927 18:38:08.608178 13311 raft_consensus.cc:2454] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8079d590210424dbdeac88bbe706045 in term 0.
I20260927 18:38:08.608222 13108 leader_election.cc:302] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41d9d386cbd44064877578ce2fe3c0e0, d8079d590210424dbdeac88bbe706045; no voters:
I20260927 18:38:08.608487 13535 raft_consensus.cc:2781] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:08.608580 13535 raft_consensus.cc:482] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:08.608629 13535 raft_consensus.cc:3037] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:08.609584 13535 raft_consensus.cc:504] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.609864 13535 leader_election.cc:288] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [CANDIDATE]: Term 1 election: Requested vote from peers 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573), 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:08.610136 13448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c49cfb514c54a669d87582e7ba36834" candidate_uuid: "d8079d590210424dbdeac88bbe706045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41d9d386cbd44064877578ce2fe3c0e0"
I20260927 18:38:08.610157 13311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c49cfb514c54a669d87582e7ba36834" candidate_uuid: "d8079d590210424dbdeac88bbe706045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a428c09443549899f21249cf969ff9c"
I20260927 18:38:08.610234 13448 raft_consensus.cc:3037] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:08.610257 13311 raft_consensus.cc:3037] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:08.611163 13311 raft_consensus.cc:2454] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8079d590210424dbdeac88bbe706045 in term 1.
I20260927 18:38:08.611167 13448 raft_consensus.cc:2454] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8079d590210424dbdeac88bbe706045 in term 1.
I20260927 18:38:08.611385 13108 leader_election.cc:302] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41d9d386cbd44064877578ce2fe3c0e0, d8079d590210424dbdeac88bbe706045; no voters:
I20260927 18:38:08.611532 13535 raft_consensus.cc:2781] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:08.611693 13535 raft_consensus.cc:686] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 1 LEADER]: Becoming Leader. State: Replica: d8079d590210424dbdeac88bbe706045, State: Running, Role: LEADER
I20260927 18:38:08.612082 13535 consensus_queue.cc:237] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:08.615396 12255 catalog_manager.cc:5495] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8079d590210424dbdeac88bbe706045 (127.6.4.1). New cstate: current_term: 1 leader_uuid: "d8079d590210424dbdeac88bbe706045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:08.643157 13523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.639994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.6.4.1:34467)
0927 18:38:08.640147 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.640152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.640165 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:08.640259 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.640261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.640313 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.640394 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:08.640572 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.641076 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.641869 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.642074 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.642148 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.642425 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:08.642869 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:08.642977 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.643007 (+ 30us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.643053 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:38:08.648602 13311 raft_consensus.cc:1261] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Refusing update from remote peer d8079d590210424dbdeac88bbe706045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:08.648602 13448 raft_consensus.cc:1261] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Refusing update from remote peer d8079d590210424dbdeac88bbe706045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:08.649290 13539 consensus_queue.cc:1035] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:08.649464 13535 consensus_queue.cc:1035] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:08.657482 13544 mvcc.cc:205] Tried to move back new op lower bound from 7334028446297944064 to 7334028446157795328. Current Snapshot: MvccSnapshot[applied={T|T < 7334028446297944064}]
I20260927 18:38:08.666023 12255 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:08.666204 12255 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:08.669878 12255 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18fff560805b45478e311f545c5fbede: Sending DeleteTablet for 3 replicas of tablet 5c49cfb514c54a669d87582e7ba36834
I20260927 18:38:08.670487 13154 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c49cfb514c54a669d87582e7ba36834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:08 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:33040
I20260927 18:38:08.670606 13291 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c49cfb514c54a669d87582e7ba36834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:08 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:41340
I20260927 18:38:08.671077 13552 tablet_replica.cc:330] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: stopping tablet replica
I20260927 18:38:08.671084 13428 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c49cfb514c54a669d87582e7ba36834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:08 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:47110
I20260927 18:38:08.671470 13552 raft_consensus.cc:2229] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:08.671847 13552 raft_consensus.cc:2258] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:08.672230 13553 tablet_replica.cc:330] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: stopping tablet replica
I20260927 18:38:08.672576 13553 raft_consensus.cc:2229] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:08.672787 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12224
I20260927 18:38:08.672857 13553 raft_consensus.cc:2258] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:08.674582 13553 ts_tablet_manager.cc:1905] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:08.676769 13553 ts_tablet_manager.cc:1918] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:08.676855 13553 log.cc:1196] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/5c49cfb514c54a669d87582e7ba36834
I20260927 18:38:08.677104 13553 ts_tablet_manager.cc:1939] T 5c49cfb514c54a669d87582e7ba36834 P 0a428c09443549899f21249cf969ff9c: Deleting consensus metadata
W20260927 18:38:08.679796 13246 connection.cc:535] server connection from 127.0.0.1:41340 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:38:08.680133 13554 tablet_replica.cc:330] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: stopping tablet replica
I20260927 18:38:08.680408 13554 raft_consensus.cc:2229] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:08.680923 13554 raft_consensus.cc:2258] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:08.681419 13552 ts_tablet_manager.cc:1905] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:08.685667 13552 ts_tablet_manager.cc:1918] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:08.685753 13552 log.cc:1196] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/5c49cfb514c54a669d87582e7ba36834
I20260927 18:38:08.686033 13552 ts_tablet_manager.cc:1939] T 5c49cfb514c54a669d87582e7ba36834 P d8079d590210424dbdeac88bbe706045: Deleting consensus metadata
W20260927 18:38:08.687150 13108 connection.cc:441] server connection from 127.0.0.1:33040 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33040 (request call id 1) could send its response
I20260927 18:38:08.688618 13554 ts_tablet_manager.cc:1905] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:08.689225 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.671818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.6.4.60:34305)
0927 18:38:08.671924 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.671929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.680350 (+ 8421us) server_negotiation.cc:407] Connection header received
0927 18:38:08.680486 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.680490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.680536 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.680583 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.681685 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.682308 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.683412 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.683708 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.683801 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.685418 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.685422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.685424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.685448 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.687225 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.687639 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.687642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.687644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.687721 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.688001 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.688004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.688006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.688211 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.688319 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.688972 (+ 653us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.689076 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:38:08.691299 13554 ts_tablet_manager.cc:1918] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:08.691442 13554 log.cc:1196] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/5c49cfb514c54a669d87582e7ba36834
I20260927 18:38:08.691790 13554 ts_tablet_manager.cc:1939] T 5c49cfb514c54a669d87582e7ba36834 P 41d9d386cbd44064877578ce2fe3c0e0: Deleting consensus metadata
W20260927 18:38:08.693204 13381 connection.cc:441] server connection from 127.0.0.1:47110 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47110 (request call id 1) could send its response
I20260927 18:38:08.696367 13509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.683931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.6.4.60:34305)
0927 18:38:08.684032 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.684036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.685661 (+ 1625us) server_negotiation.cc:407] Connection header received
0927 18:38:08.685823 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.685826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.685867 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.685924 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.687525 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.688249 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.690859 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.691199 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.691273 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.692807 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.692811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.692813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.692836 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.694822 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.695243 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.695247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.695249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.695342 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.695611 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.695614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.695616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.695779 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.695887 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.696117 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.696219 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:38:08.698025 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:08.688439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.6.4.61:42643)
0927 18:38:08.688547 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:08.688551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:08.689626 (+ 1075us) server_negotiation.cc:407] Connection header received
0927 18:38:08.689874 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:08.689878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:08.689930 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:08.689980 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:08.690649 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.691402 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.692797 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:08.693124 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:08.693203 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:08.694648 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:08.694652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:08.694655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:08.694680 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:08.696389 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.696857 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.696860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.696862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.696949 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:08.697300 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:08.697303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:08.697305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:08.697464 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:08.697568 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:08.697773 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 18:38:08.697882 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:38:09.660516 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:09.660037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:57651)
0927 18:38:09.660309 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:09.660405 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
W20260927 18:38:09.660866 13358 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:43091 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:09.661470 13564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:09.661058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:41109)
0927 18:38:09.661321 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:09.661395 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":56,"threads_started":1}
W20260927 18:38:09.661741 13221 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:43091 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:09.661803 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:09.661458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:48039)
0927 18:38:09.661643 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:09.661718 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":49,"threads_started":1}
W20260927 18:38:09.662021 13495 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:43091 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:10.169953 13566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 18fff560805b45478e311f545c5fbede)
I20260927 18:38:10.170051 13567 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 18fff560805b45478e311f545c5fbede)
I20260927 18:38:10.170156 13566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:10.170243 13567 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:10.171103 13567 leader_election.cc:288] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18fff560805b45478e311f545c5fbede (127.6.4.62:43091), 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61:42643)
I20260927 18:38:10.171123 13566 leader_election.cc:288] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18fff560805b45478e311f545c5fbede (127.6.4.62:43091), b2ce58595eff452bbeda36782b6cd1da (127.6.4.60:34305)
I20260927 18:38:10.171720 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.171255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56856)
0927 18:38:10.171534 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.171624 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":57,"threads_started":1}
I20260927 18:38:10.171751 13568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.171256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56854)
0927 18:38:10.171542 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.171613 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":53,"threads_started":1}
W20260927 18:38:10.171897 13027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
W20260927 18:38:10.171988 12839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:10.172904 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.172639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56884)
0927 18:38:10.172795 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.172842 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":29}
I20260927 18:38:10.173041 13568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.172826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56894)
0927 18:38:10.172941 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.172986 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260927 18:38:10.173074 12839 leader_election.cc:334] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
W20260927 18:38:10.173151 13027 leader_election.cc:334] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:10.175298 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.171472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:42643 (local address 127.0.0.1:48786)
0927 18:38:10.172081 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.172123 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:10.172224 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:10.172571 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:10.172574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:10.172592 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:10.172684 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:10.172691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.173564 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.173566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:10.174334 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:10.174342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.174932 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.174935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:10.175033 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:10.175051 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:38:10.175113 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:38:10.175177 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260927 18:38:10.175343 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.171439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:34305 (local address 127.0.0.1:40982)
0927 18:38:10.171922 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.171936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:10.172033 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:10.172469 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:10.172472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:10.172490 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:10.172590 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:10.172597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.173552 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.173555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:10.174336 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:10.174345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.174943 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.174946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:10.175024 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:10.175043 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 18:38:10.175106 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:38:10.175173 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":47,"threads_started":1}
I20260927 18:38:10.175356 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.171752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.6.4.61:42643)
0927 18:38:10.171969 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:10.171974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:10.172305 (+ 331us) server_negotiation.cc:407] Connection header received
0927 18:38:10.172350 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:10.172352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:10.172392 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:10.172455 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:10.172797 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.173468 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.174447 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.174836 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.174920 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:10.175012 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:10.175140 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:38:10.175191 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":65,"threads_started":1}
I20260927 18:38:10.175330 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.171739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.6.4.60:34305)
0927 18:38:10.172244 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:10.172248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:10.172261 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:10.172303 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:10.172306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:10.172349 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:10.172694 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:10.172783 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.173458 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.174447 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:10.174843 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:10.174928 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:10.175011 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:10.175126 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 18:38:10.175231 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":448,"thread_start_us":49,"threads_started":1}
I20260927 18:38:10.175942 12863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ce58595eff452bbeda36782b6cd1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9eacdc885dfb4f8d94656268bf781c60" is_pre_election: true
I20260927 18:38:10.175942 13043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9eacdc885dfb4f8d94656268bf781c60" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2ce58595eff452bbeda36782b6cd1da" is_pre_election: true
I20260927 18:38:10.176131 13043 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eacdc885dfb4f8d94656268bf781c60 in term 1.
I20260927 18:38:10.176131 12863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2ce58595eff452bbeda36782b6cd1da in term 1.
I20260927 18:38:10.176446 12837 leader_election.cc:302] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9eacdc885dfb4f8d94656268bf781c60, b2ce58595eff452bbeda36782b6cd1da; no voters: 18fff560805b45478e311f545c5fbede
I20260927 18:38:10.176461 13026 leader_election.cc:302] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9eacdc885dfb4f8d94656268bf781c60, b2ce58595eff452bbeda36782b6cd1da; no voters: 18fff560805b45478e311f545c5fbede
I20260927 18:38:10.176666 13566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:38:10.176839 13566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 18fff560805b45478e311f545c5fbede)
I20260927 18:38:10.176875 13567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:38:10.176920 13566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:38:10.176954 13567 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 18fff560805b45478e311f545c5fbede)
I20260927 18:38:10.177004 13567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:38:10.177973 13566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:10.177982 13567 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:10.178296 13566 leader_election.cc:288] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [CANDIDATE]: Term 2 election: Requested vote from peers 18fff560805b45478e311f545c5fbede (127.6.4.62:43091), b2ce58595eff452bbeda36782b6cd1da (127.6.4.60:34305)
I20260927 18:38:10.178305 13567 leader_election.cc:288] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 2 election: Requested vote from peers 18fff560805b45478e311f545c5fbede (127.6.4.62:43091), 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61:42643)
I20260927 18:38:10.178613 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.178399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56916)
0927 18:38:10.178513 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.178557 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260927 18:38:10.178612 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.178393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56908)
0927 18:38:10.178509 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.178555 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260927 18:38:10.178640 12863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ce58595eff452bbeda36782b6cd1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9eacdc885dfb4f8d94656268bf781c60"
I20260927 18:38:10.178642 13043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9eacdc885dfb4f8d94656268bf781c60" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2ce58595eff452bbeda36782b6cd1da"
I20260927 18:38:10.178795 13043 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9eacdc885dfb4f8d94656268bf781c60 in current term 2: Already voted for candidate b2ce58595eff452bbeda36782b6cd1da in this term.
I20260927 18:38:10.178797 12863 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b2ce58595eff452bbeda36782b6cd1da in current term 2: Already voted for candidate 9eacdc885dfb4f8d94656268bf781c60 in this term.
I20260927 18:38:10.179169 13568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.178960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56918)
0927 18:38:10.179076 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.179119 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260927 18:38:10.179265 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.179037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56930)
0927 18:38:10.179154 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.179211 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260927 18:38:10.179280 13027 leader_election.cc:334] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:10.179332 13027 leader_election.cc:302] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2ce58595eff452bbeda36782b6cd1da; no voters: 18fff560805b45478e311f545c5fbede, 9eacdc885dfb4f8d94656268bf781c60
W20260927 18:38:10.179342 12839 leader_election.cc:334] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:10.179387 12839 leader_election.cc:302] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9eacdc885dfb4f8d94656268bf781c60; no voters: 18fff560805b45478e311f545c5fbede, b2ce58595eff452bbeda36782b6cd1da
I20260927 18:38:10.179445 13567 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 18:38:10.179497 13566 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 18:38:10.661798 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.661376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:60661)
0927 18:38:10.661626 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.661692 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260927 18:38:10.662516 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.662155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:48111)
0927 18:38:10.662371 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.662428 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260927 18:38:10.662955 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:10.662474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:36411)
0927 18:38:10.662826 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:10.662885 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":71,"threads_started":1}
I20260927 18:38:11.662825 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.662385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:36473)
0927 18:38:11.662614 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.662718 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260927 18:38:11.663450 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.663076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:49775)
0927 18:38:11.663304 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.663362 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260927 18:38:11.663811 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.663480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:49251)
0927 18:38:11.663681 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.663733 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":55,"threads_started":1}
I20260927 18:38:11.879563 13582 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:11.879686 13582 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:11.880060 13582 leader_election.cc:288] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 18fff560805b45478e311f545c5fbede (127.6.4.62:43091), 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61:42643)
I20260927 18:38:11.880549 12863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ce58595eff452bbeda36782b6cd1da" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9eacdc885dfb4f8d94656268bf781c60" is_pre_election: true
I20260927 18:38:11.880645 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.880266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:56998)
0927 18:38:11.880481 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.880549 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
I20260927 18:38:11.880735 12863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2ce58595eff452bbeda36782b6cd1da in term 2.
I20260927 18:38:11.881057 13026 leader_election.cc:302] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eacdc885dfb4f8d94656268bf781c60, b2ce58595eff452bbeda36782b6cd1da; no voters:
I20260927 18:38:11.881234 13582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 18:38:11.881286 13582 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:11.881322 13582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:38:11.881449 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.881208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:57006)
0927 18:38:11.881297 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.881381 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260927 18:38:11.881558 13027 leader_election.cc:334] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:11.882479 13582 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:11.882822 13582 leader_election.cc:288] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 3 election: Requested vote from peers 18fff560805b45478e311f545c5fbede (127.6.4.62:43091), 9eacdc885dfb4f8d94656268bf781c60 (127.6.4.61:42643)
I20260927 18:38:11.883116 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.882909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:57016)
0927 18:38:11.883007 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.883056 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:38:11.883092 12863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2ce58595eff452bbeda36782b6cd1da" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9eacdc885dfb4f8d94656268bf781c60"
I20260927 18:38:11.883184 12863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:38:11.883666 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.883492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:57020)
0927 18:38:11.883575 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.883617 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:38:11.883751 13027 leader_election.cc:334] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
I20260927 18:38:11.884214 12863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2ce58595eff452bbeda36782b6cd1da in term 3.
I20260927 18:38:11.884438 13026 leader_election.cc:302] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9eacdc885dfb4f8d94656268bf781c60, b2ce58595eff452bbeda36782b6cd1da; no voters: 18fff560805b45478e311f545c5fbede
I20260927 18:38:11.884583 13582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:38:11.884778 13582 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [term 3 LEADER]: Becoming Leader. State: Replica: b2ce58595eff452bbeda36782b6cd1da, State: Running, Role: LEADER
I20260927 18:38:11.885012 13582 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } }
I20260927 18:38:11.886294 13585 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2ce58595eff452bbeda36782b6cd1da. Latest consensus state: current_term: 3 leader_uuid: "b2ce58595eff452bbeda36782b6cd1da" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:11.886394 13585 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:11.886636 13587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:11.888749 13587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:11.889091 13587 catalog_manager.cc:1259] Loaded cluster ID: 5cc6d271cd784999a345512aad417549
I20260927 18:38:11.889137 13587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:11.889650 13587 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:11.889985 13587 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Loaded TSK: 0
I20260927 18:38:11.890599 13587 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:38:11.968952 13041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:11.971871 13041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:11.974551 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.974320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:57036)
0927 18:38:11.974421 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.974474 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 18:38:11.974639 12863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [term 3 FOLLOWER]: Refusing update from remote peer b2ce58595eff452bbeda36782b6cd1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260927 18:38:11.975075 13585 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260927 18:38:11.975167 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.974976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.0.0.1:57040)
0927 18:38:11.975071 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.975112 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:38:11.975286 13027 consensus_peers.cc:487] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da -> Peer 18fff560805b45478e311f545c5fbede (127.6.4.62:43091): Couldn't send request to peer 18fff560805b45478e311f545c5fbede. Status: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 18:38:11.976387 13589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2ce58595eff452bbeda36782b6cd1da. Latest consensus state: current_term: 3 leader_uuid: "b2ce58595eff452bbeda36782b6cd1da" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:11.976495 13589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:11.977404 13585 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b2ce58595eff452bbeda36782b6cd1da" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:11.977505 13585 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:11.977730 13589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b2ce58595eff452bbeda36782b6cd1da" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "18fff560805b45478e311f545c5fbede" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43091 } } peers { permanent_uuid: "9eacdc885dfb4f8d94656268bf781c60" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 42643 } attrs { promote: false } } peers { permanent_uuid: "b2ce58595eff452bbeda36782b6cd1da" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 34305 } attrs { promote: false } } }
I20260927 18:38:11.977839 13589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9eacdc885dfb4f8d94656268bf781c60 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:38:11.986307 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.982471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:34467 (local address 127.0.0.1:42756)
0927 18:38:11.982568 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.982582 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:11.982669 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:11.982989 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:11.982992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:11.983010 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:11.983112 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:11.983118 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.984141 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.984144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:11.985106 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:11.985124 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.985977 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.985980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:11.986069 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:11.986094 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 18:38:11.986139 (+ 45us) client_negotiation.cc:240] Negotiation successful
0927 18:38:11.986198 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:38:11.987035 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.982563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.6.4.1:34467)
0927 18:38:11.982765 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:11.982768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:11.982780 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:11.982822 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:11.982824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:11.982865 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:11.983207 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:11.983311 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.984020 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.985224 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.985848 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.986745 (+ 897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:11.986808 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:11.986842 (+ 34us) server_negotiation.cc:299] Negotiation successful
0927 18:38:11.986911 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260927 18:38:11.987260 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.982995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33529 (local address 127.0.0.1:46974)
0927 18:38:11.983520 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.983532 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:11.983609 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:11.983966 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:11.983969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:11.983988 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:11.984068 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:11.984074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.984972 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.984975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:11.985825 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:11.985833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.986883 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.986886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:11.986976 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:11.986996 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 18:38:11.987060 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:38:11.987123 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":89,"threads_started":1}
I20260927 18:38:11.987476 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.983520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46974 (local address 127.6.4.2:33529)
0927 18:38:11.983754 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:11.983759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:11.983771 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:11.983821 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:11.983824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:11.983869 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:11.983949 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:11.984205 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.984855 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.985961 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.986780 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.986871 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:11.986929 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:11.987084 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 18:38:11.987145 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260927 18:38:11.987684 13154 tablet_service.cc:1467] Processing CreateTablet for tablet d7aadf32f765484aa69cefce3619e888 (DEFAULT_TABLE table=test-table [id=534ffb9ec05447098980bee906f92156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:11.987748 13598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.983305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.6.4.3:34573)
0927 18:38:11.983507 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:11.983511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:11.983843 (+ 332us) server_negotiation.cc:407] Connection header received
0927 18:38:11.984051 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:11.984054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:11.984141 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:11.984198 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:11.984531 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.985243 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.986536 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.987096 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.987184 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:11.987239 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:11.987559 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 18:38:11.987631 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260927 18:38:11.987913 13291 tablet_service.cc:1467] Processing CreateTablet for tablet d7aadf32f765484aa69cefce3619e888 (DEFAULT_TABLE table=test-table [id=534ffb9ec05447098980bee906f92156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:11.987957 13154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7aadf32f765484aa69cefce3619e888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:11.988044 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.983018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:34573 (local address 127.0.0.1:52856)
0927 18:38:11.983773 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.983785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:11.983898 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:11.984319 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:11.984322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:11.984336 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:11.984420 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:11.984425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.985342 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.985345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:11.986407 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:11.986416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:11.987325 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:11.987328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:11.987428 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:11.987449 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:38:11.987847 (+ 398us) client_negotiation.cc:240] Negotiation successful
0927 18:38:11.987917 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":68,"threads_started":1}
I20260927 18:38:11.988144 13291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7aadf32f765484aa69cefce3619e888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:11.988581 13428 tablet_service.cc:1467] Processing CreateTablet for tablet d7aadf32f765484aa69cefce3619e888 (DEFAULT_TABLE table=test-table [id=534ffb9ec05447098980bee906f92156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:11.988809 13428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7aadf32f765484aa69cefce3619e888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:11.990507 13600 tablet_bootstrap.cc:492] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Bootstrap starting.
I20260927 18:38:11.990731 13601 tablet_bootstrap.cc:492] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045: Bootstrap starting.
I20260927 18:38:11.991060 13602 tablet_bootstrap.cc:492] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Bootstrap starting.
I20260927 18:38:11.991384 13600 tablet_bootstrap.cc:654] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:11.991528 13601 tablet_bootstrap.cc:654] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:11.991905 13602 tablet_bootstrap.cc:654] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:11.992493 13600 tablet_bootstrap.cc:492] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: No bootstrap required, opened a new log
I20260927 18:38:11.992580 13600 ts_tablet_manager.cc:1397] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:11.992947 13602 tablet_bootstrap.cc:492] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: No bootstrap required, opened a new log
I20260927 18:38:11.992960 13600 raft_consensus.cc:348] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:11.993028 13602 ts_tablet_manager.cc:1397] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:11.993067 13600 raft_consensus.cc:374] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:11.993105 13600 raft_consensus.cc:729] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a428c09443549899f21249cf969ff9c, State: Initialized, Role: FOLLOWER
I20260927 18:38:11.993221 13600 consensus_queue.cc:260] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:11.993389 13602 raft_consensus.cc:348] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:11.993498 13602 raft_consensus.cc:374] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:11.993531 13600 ts_tablet_manager.cc:1428] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:11.993569 13602 raft_consensus.cc:729] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41d9d386cbd44064877578ce2fe3c0e0, State: Initialized, Role: FOLLOWER
I20260927 18:38:11.993727 13602 consensus_queue.cc:260] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:11.994051 13602 ts_tablet_manager.cc:1428] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:11.994405 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.993896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:33857)
0927 18:38:11.994289 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.994343 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327}
I20260927 18:38:11.995033 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.994691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:54217)
0927 18:38:11.994919 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.994970 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260927 18:38:11.995656 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.995411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:60223)
0927 18:38:11.995536 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.995585 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260927 18:38:11.996086 13494 heartbeater.cc:503] Master 127.6.4.60:34305 was elected leader, sending a full tablet report...
I20260927 18:38:11.996685 13357 heartbeater.cc:503] Master 127.6.4.60:34305 was elected leader, sending a full tablet report...
I20260927 18:38:11.996887 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.996684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:48355)
0927 18:38:11.996779 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.996826 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:38:11.997382 13601 tablet_bootstrap.cc:492] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045: No bootstrap required, opened a new log
I20260927 18:38:11.997465 13601 ts_tablet_manager.cc:1397] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260927 18:38:11.997936 13601 raft_consensus.cc:348] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:11.998024 13601 raft_consensus.cc:374] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:11.998054 13601 raft_consensus.cc:729] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8079d590210424dbdeac88bbe706045, State: Initialized, Role: FOLLOWER
I20260927 18:38:11.998165 13601 consensus_queue.cc:260] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:11.998452 13601 ts_tablet_manager.cc:1428] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:11.999008 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.998747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:50631)
0927 18:38:11.998896 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.998944 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260927 18:38:11.999691 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:11.999395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:58549)
0927 18:38:11.999575 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:11.999622 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260927 18:38:12.000272 13220 heartbeater.cc:503] Master 127.6.4.60:34305 was elected leader, sending a full tablet report...
W20260927 18:38:12.016863 13360 tablet.cc:2369] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:12.036303 13607 raft_consensus.cc:482] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:12.036422 13607 raft_consensus.cc:504] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:12.037432 13607 leader_election.cc:288] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467), 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:12.041090 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.037747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:40883 (local address 127.6.4.2:33529)
0927 18:38:12.037859 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:12.037863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:12.038112 (+ 249us) server_negotiation.cc:407] Connection header received
0927 18:38:12.038240 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:12.038244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:12.038289 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:12.038365 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:12.038655 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.039137 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.040202 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.040577 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.040672 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:12.040731 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:12.040906 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 18:38:12.040971 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:38:12.041100 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.037640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:34467 (local address 127.6.4.3:47063)
0927 18:38:12.037771 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.037787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:12.037885 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:12.038209 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:12.038212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:12.038232 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:12.038369 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:12.038376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.039148 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.039151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:12.040058 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:12.040067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.040706 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.040710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:12.040809 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:12.040832 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:38:12.040899 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:38:12.040971 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":35}
I20260927 18:38:12.041103 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.037858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:47063 (local address 127.6.4.1:34467)
0927 18:38:12.037965 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:12.037969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:12.037983 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:38:12.038058 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:12.038061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:12.038126 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:12.038195 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:12.038484 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.039029 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.040204 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.040587 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.040693 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:12.040761 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:12.040926 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:38:12.040993 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:12.041148 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.037640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33529 (local address 127.6.4.3:40883)
0927 18:38:12.038030 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.038042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:12.038142 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:12.038396 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:12.038399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:12.038415 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:12.038519 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:12.038525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.039237 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.039240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:12.040073 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:12.040081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.040689 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.040692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:12.040787 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:12.040818 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 18:38:12.040875 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:38:12.040946 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":118,"thread_start_us":109,"threads_started":1}
I20260927 18:38:12.041595 13311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7aadf32f765484aa69cefce3619e888" candidate_uuid: "41d9d386cbd44064877578ce2fe3c0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a428c09443549899f21249cf969ff9c" is_pre_election: true
I20260927 18:38:12.041596 13174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7aadf32f765484aa69cefce3619e888" candidate_uuid: "41d9d386cbd44064877578ce2fe3c0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8079d590210424dbdeac88bbe706045" is_pre_election: true
I20260927 18:38:12.041749 13311 raft_consensus.cc:2454] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41d9d386cbd44064877578ce2fe3c0e0 in term 0.
I20260927 18:38:12.041806 13174 raft_consensus.cc:2454] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41d9d386cbd44064877578ce2fe3c0e0 in term 0.
I20260927 18:38:12.042069 13381 leader_election.cc:302] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a428c09443549899f21249cf969ff9c, 41d9d386cbd44064877578ce2fe3c0e0; no voters:
I20260927 18:38:12.042320 13607 raft_consensus.cc:2781] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:12.042385 13607 raft_consensus.cc:482] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:12.042424 13607 raft_consensus.cc:3037] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:12.043447 13607 raft_consensus.cc:504] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:12.043721 13607 leader_election.cc:288] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [CANDIDATE]: Term 1 election: Requested vote from peers d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467), 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529)
I20260927 18:38:12.043982 13174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7aadf32f765484aa69cefce3619e888" candidate_uuid: "41d9d386cbd44064877578ce2fe3c0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8079d590210424dbdeac88bbe706045"
I20260927 18:38:12.044108 13174 raft_consensus.cc:3037] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:12.044142 13311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7aadf32f765484aa69cefce3619e888" candidate_uuid: "41d9d386cbd44064877578ce2fe3c0e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a428c09443549899f21249cf969ff9c"
I20260927 18:38:12.044246 13311 raft_consensus.cc:3037] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:12.045248 13174 raft_consensus.cc:2454] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41d9d386cbd44064877578ce2fe3c0e0 in term 1.
I20260927 18:38:12.045248 13311 raft_consensus.cc:2454] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41d9d386cbd44064877578ce2fe3c0e0 in term 1.
I20260927 18:38:12.045504 13384 leader_election.cc:302] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41d9d386cbd44064877578ce2fe3c0e0, d8079d590210424dbdeac88bbe706045; no voters:
I20260927 18:38:12.045651 13607 raft_consensus.cc:2781] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:12.045832 13607 raft_consensus.cc:686] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 LEADER]: Becoming Leader. State: Replica: 41d9d386cbd44064877578ce2fe3c0e0, State: Running, Role: LEADER
I20260927 18:38:12.046056 13607 consensus_queue.cc:237] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } }
I20260927 18:38:12.046406 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.046225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:41285)
0927 18:38:12.046306 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.046354 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260927 18:38:12.048179 13040 catalog_manager.cc:5495] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "41d9d386cbd44064877578ce2fe3c0e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41d9d386cbd44064877578ce2fe3c0e0" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 34573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:12.065606 13598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.062432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.6.4.3:34573)
0927 18:38:12.062550 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:12.062554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:12.062569 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:38:12.062708 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:12.062711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:12.062771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:12.062867 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:12.063032 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.063632 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.064445 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:12.064657 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:12.064725 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:12.064933 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:12.065352 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:12.065432 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:12.065473 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 18:38:12.065516 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:38:12.067881 13311 raft_consensus.cc:1261] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Refusing update from remote peer 41d9d386cbd44064877578ce2fe3c0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:12.068087 13174 raft_consensus.cc:1261] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Refusing update from remote peer 41d9d386cbd44064877578ce2fe3c0e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:12.068364 13610 consensus_queue.cc:1035] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a428c09443549899f21249cf969ff9c" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:12.068596 13607 consensus_queue.cc:1035] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8079d590210424dbdeac88bbe706045" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 34467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:12.070008 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.069767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:52789)
0927 18:38:12.069884 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.069940 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 18:38:12.070822 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.070598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:37605)
0927 18:38:12.070696 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.070757 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:38:12.071762 13613 mvcc.cc:205] Tried to move back new op lower bound from 7334028460306030592 to 7334028460223127552. Current Snapshot: MvccSnapshot[applied={T|T < 7334028460306030592}]
I20260927 18:38:12.073117 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.072782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.2:54443)
0927 18:38:12.073001 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.073057 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260927 18:38:12.073325 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.072562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:57481)
0927 18:38:12.073215 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.073265 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595}
I20260927 18:38:12.075795 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.075551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.3:53337)
0927 18:38:12.075682 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.075740 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260927 18:38:12.079208 13580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.078983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43091 (local address 127.6.4.1:33697)
0927 18:38:12.079097 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.079147 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:38:12.079353 13041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:12.079500 13041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:12.083410 13041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2ce58595eff452bbeda36782b6cd1da: Sending DeleteTablet for 3 replicas of tablet d7aadf32f765484aa69cefce3619e888
I20260927 18:38:12.084021 13154 tablet_service.cc:1514] Processing DeleteTablet for tablet d7aadf32f765484aa69cefce3619e888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:12 UTC) from {username='oryx', principal='oryx/127.6.4.60@KRBTEST.COM'} at 127.0.0.1:42756
I20260927 18:38:12.084331 13291 tablet_service.cc:1514] Processing DeleteTablet for tablet d7aadf32f765484aa69cefce3619e888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:12 UTC) from {username='oryx', principal='oryx/127.6.4.60@KRBTEST.COM'} at 127.0.0.1:46974
I20260927 18:38:12.084345 13623 tablet_replica.cc:330] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045: stopping tablet replica
I20260927 18:38:12.084455 13623 raft_consensus.cc:2229] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:12.084560 13623 raft_consensus.cc:2258] T d7aadf32f765484aa69cefce3619e888 P d8079d590210424dbdeac88bbe706045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:12.084816 13428 tablet_service.cc:1514] Processing DeleteTablet for tablet d7aadf32f765484aa69cefce3619e888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:12 UTC) from {username='oryx', principal='oryx/127.6.4.60@KRBTEST.COM'} at 127.0.0.1:52856
I20260927 18:38:12.085062 13624 tablet_replica.cc:330] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: stopping tablet replica
I20260927 18:38:12.085182 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 13000
I20260927 18:38:12.085213 13624 raft_consensus.cc:2229] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:12.085397 13624 raft_consensus.cc:2258] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:12.087064 13624 ts_tablet_manager.cc:1905] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:12.088260 13625 tablet_replica.cc:330] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: stopping tablet replica
I20260927 18:38:12.088408 13625 raft_consensus.cc:2229] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:12.088692 13625 raft_consensus.cc:2258] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:12.089272 13625 ts_tablet_manager.cc:1905] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:12.089736 13624 ts_tablet_manager.cc:1918] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:12.089861 13624 log.cc:1196] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/d7aadf32f765484aa69cefce3619e888
I20260927 18:38:12.090181 13624 ts_tablet_manager.cc:1939] T d7aadf32f765484aa69cefce3619e888 P 0a428c09443549899f21249cf969ff9c: Deleting consensus metadata
I20260927 18:38:12.090865 13025 catalog_manager.cc:4841] TS 0a428c09443549899f21249cf969ff9c (127.6.4.2:33529): tablet d7aadf32f765484aa69cefce3619e888 (table test-table [id=534ffb9ec05447098980bee906f92156]) successfully deleted
I20260927 18:38:12.091538 13625 ts_tablet_manager.cc:1918] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:12.091605 13625 log.cc:1196] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/d7aadf32f765484aa69cefce3619e888
I20260927 18:38:12.091822 13625 ts_tablet_manager.cc:1939] T d7aadf32f765484aa69cefce3619e888 P 41d9d386cbd44064877578ce2fe3c0e0: Deleting consensus metadata
I20260927 18:38:12.092535 13026 catalog_manager.cc:4841] TS 41d9d386cbd44064877578ce2fe3c0e0 (127.6.4.3:34573): tablet d7aadf32f765484aa69cefce3619e888 (table test-table [id=534ffb9ec05447098980bee906f92156]) successfully deleted
I20260927 18:38:12.093974 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 13228
I20260927 18:38:12.094121 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:12.093884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:34467 (local address 127.0.0.1:42770)
0927 18:38:12.093993 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:12.094052 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:34467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:38:12.097165 13028 catalog_manager.cc:4584] TS d8079d590210424dbdeac88bbe706045 (127.6.4.1:34467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7aadf32f765484aa69cefce3619e888: Network error: Client connection negotiation failed: client connection to 127.6.4.1:34467: connect: Connection refused (error 111)
I20260927 18:38:12.099426 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 13364
I20260927 18:38:12.104517 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12720
I20260927 18:38:12.109261 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 12907
2026-09-27T18:38:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6846 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:12 dist-test-slave-bzzj krb5kdc[13628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:12 dist-test-slave-bzzj krb5kdc[13628](info): set up 2 sockets
Sep 27 18:38:12 dist-test-slave-bzzj krb5kdc[13628](info): commencing operation
krb5kdc: starting...
W20260927 18:38:13.064278 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab.
Entry for principal oryx/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab.
I20260927 18:38:13.172959 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:37971
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab
--principal=oryx/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:37517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:37971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:13.276038 13641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:13.276306 13641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:13.276361 13641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:13.280192 13641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:13.280236 13641 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:38:13.280256 13641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:13.280280 13641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:13.280308 13641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:13.280336 13641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:13.285333 13641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:37971
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.62.keytab
--principal=oryx/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:37971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:13.286217 13641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:13.287084 13641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:13.294849 13647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.294857 13646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:13.294984 13641 server_base.cc:1034] running on GCE node
W20260927 18:38:13.294857 13649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:13.295524 13641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:13.296510 13641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:13.297662 13641 hybrid_clock.cc:648] HybridClock initialized: now 1790534293297641 us; error 29 us; skew 500 ppm
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:13.299980 13641 init.cc:377] Logged in from keytab as oryx/127.6.4.62@KRBTEST.COM (short username oryx)
I20260927 18:38:13.300778 13641 webserver.cc:460] Webserver started at http://127.6.4.62:46185/ using document root <none> and password file <none>
I20260927 18:38:13.301260 13641 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:13.301321 13641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:13.301486 13641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:13.303277 13641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "2a82e6ca9c1e425e93cbb74f77212512"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "a1f26ea9f083839475bb931d66abab64"
server_key_iv: "c5b2ab629d3043df38557066487e7d89"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.303752 13641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "2a82e6ca9c1e425e93cbb74f77212512"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "a1f26ea9f083839475bb931d66abab64"
server_key_iv: "c5b2ab629d3043df38557066487e7d89"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.306954 13641 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 18:38:13.309087 13656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.310216 13641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:13.310326 13641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "2a82e6ca9c1e425e93cbb74f77212512"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "a1f26ea9f083839475bb931d66abab64"
server_key_iv: "c5b2ab629d3043df38557066487e7d89"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.310415 13641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:13.319192 13641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:13.321573 13641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:13.321723 13641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:13.328538 13641 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:37971
I20260927 18:38:13.328547 13718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:37971 every 8 connection(s)
I20260927 18:38:13.329365 13641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:13.331758 13719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:13.336712 13719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Bootstrap starting.
I20260927 18:38:13.337433 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 13641
I20260927 18:38:13.337548 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:13.337862 6160 external_mini_cluster.cc:1428] Setting key 8bd84483daa9a9be5f91b9374c81814e
I20260927 18:38:13.339293 13719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:13.339931 13719 log.cc:824] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:13.341840 13719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: No bootstrap required, opened a new log
I20260927 18:38:13.345737 13719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a82e6ca9c1e425e93cbb74f77212512" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37971 } }
I20260927 18:38:13.345937 13719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:13.345970 13719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a82e6ca9c1e425e93cbb74f77212512, State: Initialized, Role: FOLLOWER
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:13.346370 13719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a82e6ca9c1e425e93cbb74f77212512" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37971 } }
I20260927 18:38:13.346493 13719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:13.346553 13719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:13.346644 13719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:13.347496 13719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a82e6ca9c1e425e93cbb74f77212512" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37971 } }
I20260927 18:38:13.347828 13719 leader_election.cc:302] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a82e6ca9c1e425e93cbb74f77212512; no voters:
I20260927 18:38:13.348073 13719 leader_election.cc:288] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:13.348191 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:13.348374 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [term 1 LEADER]: Becoming Leader. State: Replica: 2a82e6ca9c1e425e93cbb74f77212512, State: Running, Role: LEADER
I20260927 18:38:13.348722 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a82e6ca9c1e425e93cbb74f77212512" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37971 } }
I20260927 18:38:13.349251 13719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:13.350193 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a82e6ca9c1e425e93cbb74f77212512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a82e6ca9c1e425e93cbb74f77212512" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37971 } } }
I20260927 18:38:13.350310 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:13.350597 13726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a82e6ca9c1e425e93cbb74f77212512. Latest consensus state: current_term: 1 leader_uuid: "2a82e6ca9c1e425e93cbb74f77212512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a82e6ca9c1e425e93cbb74f77212512" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 37971 } } }
I20260927 18:38:13.350706 13726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:13.351886 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.338837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36186 (local address 127.6.4.62:37971)
0927 18:38:13.339232 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:13.339247 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:13.339278 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:38:13.339881 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:13.339897 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:13.340270 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:13.340619 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:13.341404 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.342407 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.343562 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.344015 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.344158 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.346696 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:13.346701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:13.346712 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:13.346743 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:13.348432 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.348924 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.348931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.348936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.349064 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.349385 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.349388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.349390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.349881 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:13.350071 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:13.350776 (+ 705us) server_negotiation.cc:299] Negotiation successful
0927 18:38:13.351072 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":123,"threads_started":1}
W20260927 18:38:13.353914 13739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:38:13.353983 13739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:38:13.354096 13733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:13.354740 13733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:13.359278 13733 catalog_manager.cc:1347] Generated new cluster ID: 80e90b8273c04e0489798ff8eaabd07c
I20260927 18:38:13.359333 13733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:13.396600 13733 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:13.397460 13733 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:13.404706 13733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Generated new TSK 0
I20260927 18:38:13.405452 13733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab.
Entry for principal oryx/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab.
I20260927 18:38:13.436326 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37971
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:37517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:13.540380 13745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:13.540603 13745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:13.540644 13745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:13.544809 13745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:13.544868 13745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:13.544905 13745 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:13.544937 13745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:13.550020 13745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:13.550976 13745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:13.551875 13745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:13.560776 13750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.560802 13751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.560860 13753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:13.560985 13745 server_base.cc:1034] running on GCE node
I20260927 18:38:13.561347 13745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:13.561911 13745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:13.563098 13745 hybrid_clock.cc:648] HybridClock initialized: now 1790534293563066 us; error 34 us; skew 500 ppm
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:13.565721 13745 init.cc:377] Logged in from keytab as oryx/127.6.4.1@KRBTEST.COM (short username oryx)
I20260927 18:38:13.566458 13745 webserver.cc:460] Webserver started at http://127.6.4.1:35195/ using document root <none> and password file <none>
I20260927 18:38:13.566944 13745 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:13.566996 13745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:13.567162 13745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:13.569092 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "302a820be6374d8d87174c73a2ac5be7"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "fce747ebfecb34857b4b98c64b7a24e8"
server_key_iv: "1bcbeb676d37fe9601a4503915053e57"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.569564 13745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "302a820be6374d8d87174c73a2ac5be7"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "fce747ebfecb34857b4b98c64b7a24e8"
server_key_iv: "1bcbeb676d37fe9601a4503915053e57"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.572986 13745 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:38:13.575212 13760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.576329 13745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:13.576438 13745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "302a820be6374d8d87174c73a2ac5be7"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "fce747ebfecb34857b4b98c64b7a24e8"
server_key_iv: "1bcbeb676d37fe9601a4503915053e57"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.576520 13745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:13.603250 13745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:13.605835 13745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:13.605990 13745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:13.607270 13745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:13.607316 13745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.607366 13745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:13.607395 13745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:13.616844 13745 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:36125
I20260927 18:38:13.616912 13875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:36125 every 8 connection(s)
I20260927 18:38:13.617817 13745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:13.619864 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.607985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:56595 (local address 127.6.4.62:37971)
0927 18:38:13.608086 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:13.608105 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:13.609072 (+ 967us) server_negotiation.cc:407] Connection header received
0927 18:38:13.610039 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:13.610043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:13.610096 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:13.610150 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:13.611657 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.612499 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.613825 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.614202 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.614307 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.616749 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:13.616753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:13.616755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:13.616800 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:13.618225 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.618736 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.618739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.618740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.618813 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.619045 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.619047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.619048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.619195 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:13.619260 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:13.619622 (+ 362us) server_negotiation.cc:299] Negotiation successful
0927 18:38:13.619749 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:38:13.620570 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.608484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.6.4.1:56595)
0927 18:38:13.608934 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:13.608972 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:13.609809 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:13.610537 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:13.610546 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:13.610901 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:13.611507 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.611524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.612620 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.612624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.613724 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.613734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.614302 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.614342 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.614468 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.614995 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:13.615016 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:13.616567 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:13.618395 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.618401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.618413 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.618663 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.618882 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.618884 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.618885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.618992 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.619260 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:13.619264 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:13.619453 (+ 189us) client_negotiation.cc:769] Sending connection context
0927 18:38:13.619588 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 18:38:13.619807 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":134,"threads_started":1}
I20260927 18:38:13.621272 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 13745
I20260927 18:38:13.621397 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:13.621670 6160 external_mini_cluster.cc:1428] Setting key d6cd6dc1d4e11eaf5161b2ec61500ec2
I20260927 18:38:13.621702 13876 heartbeater.cc:347] Connected to a master server at 127.6.4.62:37971
I20260927 18:38:13.621860 13876 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:13.622327 13876 heartbeater.cc:511] Master 127.6.4.62:37971 requested a full tablet report, sending...
I20260927 18:38:13.624858 13672 ts_manager.cc:194] Registered new tserver with Master: 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125)
I20260927 18:38:13.625881 13672 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:56595
WARNING: no policy specified for oryx/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:38:13.632655 13882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.623617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.6.4.1:42657)
0927 18:38:13.626444 (+ 2827us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:13.626456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:13.626567 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:13.626855 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:13.626857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:13.627039 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:13.627147 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.627154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.627842 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.627845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.628721 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.628732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.629128 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.629144 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.629234 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.629630 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:13.629643 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:13.630007 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:13.631418 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.631421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.631424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.631588 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.631820 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.631822 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.631823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.631885 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.632305 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:13.632308 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:13.632378 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 18:38:13.632452 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:38:13.632541 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2753,"thread_start_us":96,"threads_started":1}
I20260927 18:38:13.632695 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.623703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:42657 (local address 127.6.4.62:37971)
0927 18:38:13.623814 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:13.623818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:13.626507 (+ 2689us) server_negotiation.cc:407] Connection header received
0927 18:38:13.626695 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:13.626699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:13.626747 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:13.626829 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:13.627250 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.627723 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.628850 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.629045 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.629117 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.630124 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:13.630127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:13.630129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:13.630148 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:13.631327 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.631673 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.631675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.631676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.631744 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.631987 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.631990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.631992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.632214 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:13.632291 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:13.632496 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 18:38:13.632578 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab.
Entry for principal oryx/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab.
I20260927 18:38:13.646276 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37971
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:37517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:13.746577 13885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:13.746778 13885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:13.746819 13885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:13.750732 13885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:13.750778 13885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:13.750823 13885 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:13.750872 13885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:13.755725 13885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:13.756603 13885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:13.757479 13885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:13.765784 13891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.765889 13890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.765873 13893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:13.766005 13885 server_base.cc:1034] running on GCE node
I20260927 18:38:13.766419 13885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:13.766961 13885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:13.768123 13885 hybrid_clock.cc:648] HybridClock initialized: now 1790534293768090 us; error 44 us; skew 500 ppm
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:13.770550 13885 init.cc:377] Logged in from keytab as oryx/127.6.4.2@KRBTEST.COM (short username oryx)
I20260927 18:38:13.771299 13885 webserver.cc:460] Webserver started at http://127.6.4.2:45639/ using document root <none> and password file <none>
I20260927 18:38:13.771790 13885 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:13.771850 13885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:13.772015 13885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:13.773850 13885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "eb24b63588ca44898128ca98bb15083b"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "ad1c69e5e380096f29017e1f4cd748a6"
server_key_iv: "e914d69356d9ceb6e7cf3971f8c2557b"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.774350 13885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "eb24b63588ca44898128ca98bb15083b"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "ad1c69e5e380096f29017e1f4cd748a6"
server_key_iv: "e914d69356d9ceb6e7cf3971f8c2557b"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.777599 13885 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:13.779727 13900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.780857 13885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:13.780947 13885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "eb24b63588ca44898128ca98bb15083b"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "ad1c69e5e380096f29017e1f4cd748a6"
server_key_iv: "e914d69356d9ceb6e7cf3971f8c2557b"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.781036 13885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:13.791615 13885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:13.794317 13885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:13.794458 13885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:13.795686 13885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:13.795732 13885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.795768 13885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:13.795794 13885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:13.805027 13885 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:35631
I20260927 18:38:13.805095 14015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:35631 every 8 connection(s)
I20260927 18:38:13.805945 13885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:13.808071 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.796439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:49455 (local address 127.6.4.62:37971)
0927 18:38:13.796581 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:13.796585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:13.797301 (+ 716us) server_negotiation.cc:407] Connection header received
0927 18:38:13.798416 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:13.798420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:13.798475 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:13.798532 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:13.800071 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.800896 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.802070 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.802408 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.802514 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.804949 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:13.804952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:13.804954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:13.804978 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:13.806304 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.806827 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.806830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.806831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.806909 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.807172 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.807174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.807175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.807326 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:13.807395 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:13.807861 (+ 466us) server_negotiation.cc:299] Negotiation successful
0927 18:38:13.807957 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:38:13.808933 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.796728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.6.4.2:49455)
0927 18:38:13.797156 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:13.797202 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:13.798182 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:13.798913 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:13.798922 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:13.799283 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:13.799892 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.799910 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.801033 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.801037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.801964 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.801973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.802505 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.802545 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.802688 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.803211 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:13.803227 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:13.804760 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:13.806428 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.806433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.806445 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.806731 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.806974 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.806975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.806977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.807099 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.807436 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:13.807441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:13.807662 (+ 221us) client_negotiation.cc:769] Sending connection context
0927 18:38:13.807820 (+ 158us) client_negotiation.cc:240] Negotiation successful
0927 18:38:13.808036 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260927 18:38:13.810055 14016 heartbeater.cc:347] Connected to a master server at 127.6.4.62:37971
I20260927 18:38:13.810186 14016 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:13.810608 14016 heartbeater.cc:511] Master 127.6.4.62:37971 requested a full tablet report, sending...
I20260927 18:38:13.810946 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 13885
I20260927 18:38:13.811072 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:13.811390 6160 external_mini_cluster.cc:1428] Setting key 873643cfc9aa2345032b543566fd628c
I20260927 18:38:13.811570 13673 ts_manager.cc:194] Registered new tserver with Master: eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:13.812266 13673 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:49455
I20260927 18:38:13.818905 14022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.812258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.6.4.2:55803)
0927 18:38:13.812558 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:13.812571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:13.812661 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:13.812870 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:13.812873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:13.813059 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:13.813163 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.813171 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.814073 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.814076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.814658 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.814676 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.815195 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.815209 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.815277 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.815678 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:13.815689 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:13.815971 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:13.817636 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.817639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.817640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.817817 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.818066 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.818068 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.818069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.818131 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.818546 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:13.818548 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:13.818614 (+ 66us) client_negotiation.cc:769] Sending connection context
0927 18:38:13.818683 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:38:13.818773 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260927 18:38:13.818990 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.812289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:55803 (local address 127.6.4.62:37971)
0927 18:38:13.812381 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:13.812385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:13.812640 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:38:13.812728 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:13.812731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:13.812777 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:13.812848 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:13.813281 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.813957 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.814795 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.815103 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.815196 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.816125 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:13.816129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:13.816132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:13.816155 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:13.817537 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.817904 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.817907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.817909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.817989 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.818205 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.818208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.818210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.818465 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:13.818565 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:13.818751 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 18:38:13.818853 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for oryx/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab.
Entry for principal oryx/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab.
I20260927 18:38:13.836447 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:37971
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:37517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:13.938390 14025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:13.938607 14025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:13.938663 14025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:13.942520 14025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:13.942571 14025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:13.942623 14025 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:38:13.942675 14025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:13.947505 14025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.4.62:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:13.948360 14025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:13.949241 14025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:13.957804 14030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.957881 14031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:13.957834 14033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:13.957916 14025 server_base.cc:1034] running on GCE node
I20260927 18:38:13.958375 14025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:13.958910 14025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:13.960060 14025 hybrid_clock.cc:648] HybridClock initialized: now 1790534293960045 us; error 25 us; skew 500 ppm
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:13.962358 14025 init.cc:377] Logged in from keytab as oryx/127.6.4.3@KRBTEST.COM (short username oryx)
I20260927 18:38:13.963068 14025 webserver.cc:460] Webserver started at http://127.6.4.3:34583/ using document root <none> and password file <none>
I20260927 18:38:13.963526 14025 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:13.963585 14025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:13.963744 14025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:13.965582 14025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "a9bc61d47431403ab7b74596cc2ba3c5"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "a64e3af4085c3a27ba27cf2848410960"
server_key_iv: "e80d44ab884516af8953007fda971dd6"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.966063 14025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "a9bc61d47431403ab7b74596cc2ba3c5"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "a64e3af4085c3a27ba27cf2848410960"
server_key_iv: "e80d44ab884516af8953007fda971dd6"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.969255 14025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260927 18:38:13.971315 14040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.972404 14025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:13.972498 14025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "a9bc61d47431403ab7b74596cc2ba3c5"
format_stamp: "Formatted at 2026-09-27 18:38:13 on dist-test-slave-bzzj"
server_key: "a64e3af4085c3a27ba27cf2848410960"
server_key_iv: "e80d44ab884516af8953007fda971dd6"
server_key_version: "encryptionkey@0"
I20260927 18:38:13.972577 14025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:13.982710 14025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:13.985146 14025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:13.985288 14025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:13.986472 14025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:13.986516 14025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:13.986563 14025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:13.986591 14025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:38:13 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:13.995993 14025 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37547
I20260927 18:38:13.996254 14155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37547 every 8 connection(s)
I20260927 18:38:13.996832 14025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:13.998880 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.987142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:38639 (local address 127.6.4.62:37971)
0927 18:38:13.987246 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:13.987250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:13.988140 (+ 890us) server_negotiation.cc:407] Connection header received
0927 18:38:13.988973 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:13.988976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:13.989034 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:13.989107 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:13.990440 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.991242 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.992733 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.993069 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.993175 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.995469 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:13.995473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:13.995476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:13.995505 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:13.996991 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.997523 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.997526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.997528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.997603 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:13.997898 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:13.997901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:13.997903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:13.998075 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:13.998159 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:13.998599 (+ 440us) server_negotiation.cc:299] Negotiation successful
0927 18:38:13.998715 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:38:13.999495 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:13.987586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.6.4.3:38639)
0927 18:38:13.987998 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:13.988034 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:13.988791 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:13.989365 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:13.989375 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:13.989769 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:13.990288 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.990306 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.991381 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.991385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.992630 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:13.992639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:13.993196 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:13.993253 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:13.993428 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:13.993910 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:13.993925 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:13.995310 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:13.997099 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.997104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.997117 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.997410 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.997686 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:13.997689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:13.997691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:13.997814 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:13.998167 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:13.998173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:13.998382 (+ 209us) client_negotiation.cc:769] Sending connection context
0927 18:38:13.998529 (+ 147us) client_negotiation.cc:240] Negotiation successful
0927 18:38:13.998793 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
I20260927 18:38:14.000504 14156 heartbeater.cc:347] Connected to a master server at 127.6.4.62:37971
I20260927 18:38:14.000629 14156 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:14.000880 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14025
I20260927 18:38:14.000973 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:14.001117 14156 heartbeater.cc:511] Master 127.6.4.62:37971 requested a full tablet report, sending...
I20260927 18:38:14.001232 6160 external_mini_cluster.cc:1428] Setting key 8c6410de2276100d900de502626b234a
I20260927 18:38:14.002193 13673 ts_manager.cc:194] Registered new tserver with Master: a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547)
I20260927 18:38:14.002775 13673 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:38639
I20260927 18:38:14.002965 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:38:14.011082 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.004699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:53739 (local address 127.6.4.62:37971)
0927 18:38:14.004818 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.004822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.004836 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:38:14.004867 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.004870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.004909 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.004985 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:14.005349 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.005968 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.006995 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.007195 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.007270 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.008256 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:14.008260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:14.008262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:14.008286 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:14.009548 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.009995 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.009998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.009999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.010084 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.010359 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.010361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.010363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.010510 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:14.010580 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.010830 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.010977 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:38:14.011099 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.003165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.6.4.3:53739)
0927 18:38:14.003423 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.003437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.003517 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.005014 (+ 1497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.005017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.005167 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:14.005257 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.005262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.006080 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.006083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.006880 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.006890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.007289 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.007306 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.007395 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.007779 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:14.007793 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:14.008130 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:14.009647 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:14.009650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:14.009653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:14.009907 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:14.010186 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:14.010188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:14.010190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:14.010285 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:14.010590 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:14.010593 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:14.010679 (+ 86us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.010785 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.010910 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:38:14.025094 13722 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:14.020036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.6.4.62:37971)
0927 18:38:14.020155 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.020158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.020168 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:38:14.020195 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.020197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.020232 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.020290 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:14.020870 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.021326 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.022085 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.022275 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.022341 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.024975 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36192: BlockingRecv error: recv got EOF from 127.0.0.1:36192 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:14.032881 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.026285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.6.4.62:37971)
0927 18:38:14.026395 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.026399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.026526 (+ 127us) server_negotiation.cc:407] Connection header received
0927 18:38:14.026623 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.026626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.026661 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.026719 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:14.027260 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.027714 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.028487 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.028684 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.028765 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.030670 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:14.030673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:14.030674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:14.030694 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:14.031692 (+ 998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.032032 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.032034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.032035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.032112 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.032316 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.032318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.032319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.032459 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:14.032559 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.032693 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.032768 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:38:14.035185 13673 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:14.038048 13673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:14.051038 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.045613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.6.4.3:37547)
0927 18:38:14.045999 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.046009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.046490 (+ 481us) server_negotiation.cc:407] Connection header received
0927 18:38:14.046565 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.046570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.046723 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.046875 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.047209 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.048071 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.049696 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.050369 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.050466 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.050624 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.050768 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.050867 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":82,"threads_started":1}
I20260927 18:38:14.051035 14179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.045468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37547 (local address 127.0.0.1:37284)
0927 18:38:14.046276 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.046289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.046377 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.047009 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.047011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.047025 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.047097 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.047103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.048184 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.048187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.049574 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.049583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.050477 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.050481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.050580 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.050617 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.050733 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.050845 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":26,"thread_start_us":80,"threads_started":1}
I20260927 18:38:14.051193 14178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.045370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:35631 (local address 127.0.0.1:60488)
0927 18:38:14.046021 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.046067 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.046182 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.046895 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.046900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.046922 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.047031 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.047038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.048329 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.048332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.049733 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.049741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.050837 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.050841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.050926 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.050949 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.050996 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.051062 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":23,"thread_start_us":55,"threads_started":1}
I20260927 18:38:14.051312 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.046092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36125 (local address 127.0.0.1:59612)
0927 18:38:14.047209 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.047221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.047306 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.047694 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.047697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.047714 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.047812 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.047817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.048892 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.048896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.049883 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.049893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.051009 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.051012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.051086 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.051104 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.051161 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.051214 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1063,"thread_start_us":92,"threads_started":1}
I20260927 18:38:14.051704 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.046244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.6.4.1:36125)
0927 18:38:14.046825 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.046838 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.047305 (+ 467us) server_negotiation.cc:407] Connection header received
0927 18:38:14.047403 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.047408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.047572 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.047705 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.047909 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.048761 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.050008 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.050708 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.050814 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.050957 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.051492 (+ 535us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.051578 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
I20260927 18:38:14.051733 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.045501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.6.4.2:35631)
0927 18:38:14.046123 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.046138 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.046163 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:38:14.046361 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.046367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.046567 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.046692 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.047141 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.048216 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.049869 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.050708 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.051345 (+ 637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.051505 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.051547 (+ 42us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.051616 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":87,"threads_started":1}
I20260927 18:38:14.054055 13950 tablet_service.cc:1467] Processing CreateTablet for tablet b2bc722007eb4bcfa57c6994f1444600 (DEFAULT_TABLE table=test-table [id=a8904c375d1e4567a208ad7368735ab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.054188 14090 tablet_service.cc:1467] Processing CreateTablet for tablet b2bc722007eb4bcfa57c6994f1444600 (DEFAULT_TABLE table=test-table [id=a8904c375d1e4567a208ad7368735ab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.054548 13810 tablet_service.cc:1467] Processing CreateTablet for tablet b2bc722007eb4bcfa57c6994f1444600 (DEFAULT_TABLE table=test-table [id=a8904c375d1e4567a208ad7368735ab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.054880 13950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bc722007eb4bcfa57c6994f1444600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.055132 14090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bc722007eb4bcfa57c6994f1444600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.055348 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bc722007eb4bcfa57c6994f1444600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.062173 14184 tablet_bootstrap.cc:492] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Bootstrap starting.
I20260927 18:38:14.062173 14185 tablet_bootstrap.cc:492] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Bootstrap starting.
I20260927 18:38:14.064365 14185 tablet_bootstrap.cc:654] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.064365 14184 tablet_bootstrap.cc:654] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.064373 14186 tablet_bootstrap.cc:492] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Bootstrap starting.
I20260927 18:38:14.065201 14184 log.cc:824] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:14.065201 14185 log.cc:824] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:14.066051 14186 tablet_bootstrap.cc:654] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.066659 14186 log.cc:824] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:14.067052 14184 tablet_bootstrap.cc:492] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: No bootstrap required, opened a new log
I20260927 18:38:14.067052 14185 tablet_bootstrap.cc:492] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: No bootstrap required, opened a new log
I20260927 18:38:14.067233 14184 ts_tablet_manager.cc:1397] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260927 18:38:14.067233 14185 ts_tablet_manager.cc:1397] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:38:14.068425 14186 tablet_bootstrap.cc:492] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: No bootstrap required, opened a new log
I20260927 18:38:14.068588 14186 ts_tablet_manager.cc:1397] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260927 18:38:14.070935 14185 raft_consensus.cc:348] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.070935 14184 raft_consensus.cc:348] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.071182 14185 raft_consensus.cc:374] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.071180 14184 raft_consensus.cc:374] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.071234 14184 raft_consensus.cc:729] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb24b63588ca44898128ca98bb15083b, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.071234 14185 raft_consensus.cc:729] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.071130 14186 raft_consensus.cc:348] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.071350 14186 raft_consensus.cc:374] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.071403 14186 raft_consensus.cc:729] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302a820be6374d8d87174c73a2ac5be7, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.071652 14185 consensus_queue.cc:260] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.071653 14184 consensus_queue.cc:260] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.071748 14186 consensus_queue.cc:260] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.072338 13876 heartbeater.cc:503] Master 127.6.4.62:37971 was elected leader, sending a full tablet report...
I20260927 18:38:14.072597 14184 ts_tablet_manager.cc:1428] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 18:38:14.072597 14185 ts_tablet_manager.cc:1428] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260927 18:38:14.072614 14186 ts_tablet_manager.cc:1428] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:38:14.072893 14016 heartbeater.cc:503] Master 127.6.4.62:37971 was elected leader, sending a full tablet report...
I20260927 18:38:14.073285 14156 heartbeater.cc:503] Master 127.6.4.62:37971 was elected leader, sending a full tablet report...
I20260927 18:38:14.094236 14191 raft_consensus.cc:482] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:14.094374 14191 raft_consensus.cc:504] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.095294 14191 leader_election.cc:288] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631), 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125)
I20260927 18:38:14.099201 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.095469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:35631 (local address 127.6.4.3:39427)
0927 18:38:14.095616 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.095630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.095705 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.095935 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.095938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.095952 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.096052 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.096057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.097005 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.097008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.098221 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.098232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.098892 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.098895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.098970 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.098989 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.099042 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.099105 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":1}
I20260927 18:38:14.099241 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.095562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:39427 (local address 127.6.4.2:35631)
0927 18:38:14.095656 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.095660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.095673 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:14.095809 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.095812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.095846 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.095910 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.096154 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.096911 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.098334 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.098791 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.098888 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.098945 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.099065 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.099129 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:38:14.099762 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.095565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:41583 (local address 127.6.4.1:36125)
0927 18:38:14.095666 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.095670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.095895 (+ 225us) server_negotiation.cc:407] Connection header received
0927 18:38:14.095998 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.096000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.096037 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.096105 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.096345 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.097042 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.098667 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.099192 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.099306 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.099360 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.099616 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.099680 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:38:14.099767 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.095519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36125 (local address 127.6.4.3:41583)
0927 18:38:14.095815 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.095825 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.095911 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.096138 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.096141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.096156 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.096244 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.096249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.097119 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.097122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.098571 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.098579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.099436 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.099439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.099511 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.099528 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.099588 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.099647 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":111,"threads_started":1}
I20260927 18:38:14.099840 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc722007eb4bcfa57c6994f1444600" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b" is_pre_election: true
I20260927 18:38:14.100086 13970 raft_consensus.cc:2454] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:14.100292 13830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc722007eb4bcfa57c6994f1444600" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7" is_pre_election: true
I20260927 18:38:14.100528 13830 raft_consensus.cc:2454] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:14.100557 14044 leader_election.cc:302] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bc61d47431403ab7b74596cc2ba3c5, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:14.100805 14191 raft_consensus.cc:2781] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:14.100893 14191 raft_consensus.cc:482] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:14.100946 14191 raft_consensus.cc:3037] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.101877 14191 raft_consensus.cc:504] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.102145 14191 leader_election.cc:288] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Requested vote from peers eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631), 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125)
I20260927 18:38:14.102428 13830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc722007eb4bcfa57c6994f1444600" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7"
I20260927 18:38:14.102447 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc722007eb4bcfa57c6994f1444600" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b"
I20260927 18:38:14.102533 13970 raft_consensus.cc:3037] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.102533 13830 raft_consensus.cc:3037] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.103627 13830 raft_consensus.cc:2454] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:14.103627 13970 raft_consensus.cc:2454] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:14.103880 14044 leader_election.cc:302] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bc61d47431403ab7b74596cc2ba3c5, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:14.104027 14191 raft_consensus.cc:2781] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:14.104243 14191 raft_consensus.cc:686] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Becoming Leader. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Running, Role: LEADER
I20260927 18:38:14.104660 14191 consensus_queue.cc:237] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } }
I20260927 18:38:14.107329 13672 catalog_manager.cc:5495] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } health_report { overall_health: UNKNOWN } } }
W20260927 18:38:14.118819 13877 tablet.cc:2369] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:14.127166 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.124166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.6.4.3:37547)
0927 18:38:14.124277 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.124281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.124310 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:38:14.124439 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.124441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.124487 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.124562 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:14.124738 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.125218 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.126022 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.126218 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.126296 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.126482 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.126842 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.126920 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.127007 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.127051 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:38:14.132153 13830 raft_consensus.cc:1261] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:14.132421 13970 raft_consensus.cc:1261] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:14.132825 14194 consensus_queue.cc:1035] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:14.132972 14191 consensus_queue.cc:1035] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:14.146425 13672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:14.146556 13672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:14.148718 13672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Sending DeleteTablet for 3 replicas of tablet b2bc722007eb4bcfa57c6994f1444600
I20260927 18:38:14.149420 13950 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bc722007eb4bcfa57c6994f1444600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:14 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60488
I20260927 18:38:14.149492 13810 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bc722007eb4bcfa57c6994f1444600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:14 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:59612
I20260927 18:38:14.149866 14207 tablet_replica.cc:330] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: stopping tablet replica
I20260927 18:38:14.149956 13672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:38:14.150148 14090 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bc722007eb4bcfa57c6994f1444600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:14 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:37284
W20260927 18:38:14.150349 13672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:14.150523 14209 tablet_replica.cc:330] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: stopping tablet replica
I20260927 18:38:14.150507 14207 raft_consensus.cc:2229] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:14.150761 14209 raft_consensus.cc:2229] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:14.150918 14207 raft_consensus.cc:2258] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:14.151079 14209 raft_consensus.cc:2258] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:14.152411 14207 ts_tablet_manager.cc:1905] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:14.153520 14209 ts_tablet_manager.cc:1905] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:14.154970 14207 ts_tablet_manager.cc:1918] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:14.155112 14207 log.cc:1196] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/b2bc722007eb4bcfa57c6994f1444600
I20260927 18:38:14.155144 13950 tablet_service.cc:1467] Processing CreateTablet for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 (DEFAULT_TABLE table=test-table [id=68f90c58df704856ac2380b0b2e989bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.155387 13950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.155575 14207 ts_tablet_manager.cc:1939] T b2bc722007eb4bcfa57c6994f1444600 P eb24b63588ca44898128ca98bb15083b: Deleting consensus metadata
I20260927 18:38:14.155967 14209 ts_tablet_manager.cc:1918] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:14.156057 14209 log.cc:1196] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/b2bc722007eb4bcfa57c6994f1444600
I20260927 18:38:14.156342 14209 ts_tablet_manager.cc:1939] T b2bc722007eb4bcfa57c6994f1444600 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting consensus metadata
I20260927 18:38:14.156962 13660 catalog_manager.cc:4841] TS eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631): tablet b2bc722007eb4bcfa57c6994f1444600 (table test-table [id=a8904c375d1e4567a208ad7368735ab3]) successfully deleted
I20260927 18:38:14.157286 13658 catalog_manager.cc:4841] TS a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547): tablet b2bc722007eb4bcfa57c6994f1444600 (table test-table [id=a8904c375d1e4567a208ad7368735ab3]) successfully deleted
I20260927 18:38:14.157388 14184 tablet_bootstrap.cc:492] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Bootstrap starting.
I20260927 18:38:14.157552 14090 tablet_service.cc:1467] Processing CreateTablet for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 (DEFAULT_TABLE table=test-table [id=68f90c58df704856ac2380b0b2e989bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.157778 14090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.158244 14184 tablet_bootstrap.cc:654] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.159181 14184 tablet_bootstrap.cc:492] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: No bootstrap required, opened a new log
I20260927 18:38:14.159255 14184 ts_tablet_manager.cc:1397] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:14.159610 14184 raft_consensus.cc:348] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.159713 14185 tablet_bootstrap.cc:492] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Bootstrap starting.
I20260927 18:38:14.159713 14184 raft_consensus.cc:374] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.159745 14184 raft_consensus.cc:729] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb24b63588ca44898128ca98bb15083b, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.159861 14184 consensus_queue.cc:260] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.160112 14184 ts_tablet_manager.cc:1428] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:14.160599 14185 tablet_bootstrap.cc:654] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.161697 14185 tablet_bootstrap.cc:492] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: No bootstrap required, opened a new log
I20260927 18:38:14.161768 14185 ts_tablet_manager.cc:1397] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:38:14.161738 13810 tablet_service.cc:1467] Processing CreateTablet for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 (DEFAULT_TABLE table=test-table [id=68f90c58df704856ac2380b0b2e989bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.161952 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.162155 14185 raft_consensus.cc:348] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.162256 14185 raft_consensus.cc:374] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.162286 14185 raft_consensus.cc:729] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.162389 14185 consensus_queue.cc:260] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.162604 14185 ts_tablet_manager.cc:1428] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:38:14.163888 14208 tablet_replica.cc:330] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: stopping tablet replica
I20260927 18:38:14.164011 14186 tablet_bootstrap.cc:492] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Bootstrap starting.
I20260927 18:38:14.164220 14208 raft_consensus.cc:2229] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:14.164522 14208 raft_consensus.cc:2258] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:14.164873 14186 tablet_bootstrap.cc:654] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.165867 14208 ts_tablet_manager.cc:1905] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:14.166735 14186 tablet_bootstrap.cc:492] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: No bootstrap required, opened a new log
I20260927 18:38:14.166814 14186 ts_tablet_manager.cc:1397] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:38:14.167203 14186 raft_consensus.cc:348] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.167307 14186 raft_consensus.cc:374] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.167336 14186 raft_consensus.cc:729] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302a820be6374d8d87174c73a2ac5be7, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.167438 14186 consensus_queue.cc:260] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.167680 14186 ts_tablet_manager.cc:1428] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:14.168048 14208 ts_tablet_manager.cc:1918] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:14.168146 14208 log.cc:1196] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/b2bc722007eb4bcfa57c6994f1444600
I20260927 18:38:14.168413 14208 ts_tablet_manager.cc:1939] T b2bc722007eb4bcfa57c6994f1444600 P 302a820be6374d8d87174c73a2ac5be7: Deleting consensus metadata
I20260927 18:38:14.169502 13657 catalog_manager.cc:4841] TS 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125): tablet b2bc722007eb4bcfa57c6994f1444600 (table test-table [id=a8904c375d1e4567a208ad7368735ab3]) successfully deleted
W20260927 18:38:14.247648 14157 tablet.cc:2369] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:14.267174 14204 raft_consensus.cc:482] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:14.267302 14204 raft_consensus.cc:504] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.267760 14204 leader_election.cc:288] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:14.268177 13830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1027f6472f4a8f9e47f0fed1d5d1d0" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7" is_pre_election: true
I20260927 18:38:14.268325 13830 raft_consensus.cc:2454] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:14.268584 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1027f6472f4a8f9e47f0fed1d5d1d0" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b" is_pre_election: true
I20260927 18:38:14.268708 14041 leader_election.cc:302] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302a820be6374d8d87174c73a2ac5be7, a9bc61d47431403ab7b74596cc2ba3c5; no voters:
I20260927 18:38:14.268743 13970 raft_consensus.cc:2454] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:14.268884 14204 raft_consensus.cc:2781] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:14.268942 14204 raft_consensus.cc:482] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:14.268978 14204 raft_consensus.cc:3037] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.269897 14204 raft_consensus.cc:504] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.270177 14204 leader_election.cc:288] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Requested vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:14.270465 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1027f6472f4a8f9e47f0fed1d5d1d0" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b"
I20260927 18:38:14.270533 13830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c1027f6472f4a8f9e47f0fed1d5d1d0" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7"
I20260927 18:38:14.270587 13970 raft_consensus.cc:3037] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.270646 13830 raft_consensus.cc:3037] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.271615 13970 raft_consensus.cc:2454] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:14.271615 13830 raft_consensus.cc:2454] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:14.271862 14044 leader_election.cc:302] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bc61d47431403ab7b74596cc2ba3c5, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:14.272051 14204 raft_consensus.cc:2781] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:14.272154 14204 raft_consensus.cc:686] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Becoming Leader. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Running, Role: LEADER
I20260927 18:38:14.272285 14204 consensus_queue.cc:237] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.273715 13673 catalog_manager.cc:5495] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } health_report { overall_health: HEALTHY } } }
W20260927 18:38:14.307080 14017 tablet.cc:2369] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534293, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.62@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:14.313189 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.305528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:37971 (local address 127.0.0.1:36212)
0927 18:38:14.305780 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.305793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.305889 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.306088 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.306091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.306471 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:14.306556 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.306562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.307440 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.307443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.308009 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.308018 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.308415 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.308427 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.308487 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.308958 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:14.308969 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:14.310328 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:14.311895 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:14.311901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:14.311906 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:14.312131 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:14.312330 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:14.312331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:14.312333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:14.312401 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:14.312720 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:14.312726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:14.312829 (+ 103us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.312915 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.313026 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
I20260927 18:38:14.313196 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.305604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.6.4.62:37971)
0927 18:38:14.305749 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.305754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.305851 (+ 97us) server_negotiation.cc:407] Connection header received
0927 18:38:14.305964 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.305967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.306004 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.306064 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:14.306668 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.307305 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.308133 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.308343 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.308420 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.310465 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:14.310469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:14.310471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:14.310492 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:14.311815 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.312202 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.312204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.312206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.312271 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.312458 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.312460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.312461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.312615 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:14.312717 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.312960 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.313076 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":68}
I20260927 18:38:14.316589 13673 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:36212:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:38:14.317221 13673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:14.321491 14090 tablet_service.cc:1467] Processing CreateTablet for tablet a165bfcad1c0487bac061d68b1ad8a4c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7be93e6210844b41a38f80293abeb0c2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:14.321492 13950 tablet_service.cc:1467] Processing CreateTablet for tablet a165bfcad1c0487bac061d68b1ad8a4c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7be93e6210844b41a38f80293abeb0c2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:14.321508 13810 tablet_service.cc:1467] Processing CreateTablet for tablet a165bfcad1c0487bac061d68b1ad8a4c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7be93e6210844b41a38f80293abeb0c2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:38:14.321854 13950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a165bfcad1c0487bac061d68b1ad8a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.321867 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a165bfcad1c0487bac061d68b1ad8a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.321846 14090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a165bfcad1c0487bac061d68b1ad8a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.324208 14186 tablet_bootstrap.cc:492] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: Bootstrap starting.
I20260927 18:38:14.324390 14185 tablet_bootstrap.cc:492] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: Bootstrap starting.
I20260927 18:38:14.324649 14184 tablet_bootstrap.cc:492] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: Bootstrap starting.
I20260927 18:38:14.325148 14186 tablet_bootstrap.cc:654] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.325155 14185 tablet_bootstrap.cc:654] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.325382 14184 tablet_bootstrap.cc:654] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.326349 14186 tablet_bootstrap.cc:492] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: No bootstrap required, opened a new log
I20260927 18:38:14.326436 14186 ts_tablet_manager.cc:1397] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:14.326463 14184 tablet_bootstrap.cc:492] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: No bootstrap required, opened a new log
I20260927 18:38:14.326542 14184 ts_tablet_manager.cc:1397] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:14.326558 14185 tablet_bootstrap.cc:492] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: No bootstrap required, opened a new log
I20260927 18:38:14.326622 14185 ts_tablet_manager.cc:1397] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260927 18:38:14.326861 14186 raft_consensus.cc:348] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.326910 14184 raft_consensus.cc:348] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.326973 14186 raft_consensus.cc:374] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.326995 14184 raft_consensus.cc:374] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.327004 14186 raft_consensus.cc:729] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302a820be6374d8d87174c73a2ac5be7, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.326963 14185 raft_consensus.cc:348] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.327034 14184 raft_consensus.cc:729] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb24b63588ca44898128ca98bb15083b, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.327062 14185 raft_consensus.cc:374] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.327098 14185 raft_consensus.cc:729] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.327118 14186 consensus_queue.cc:260] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.327149 14184 consensus_queue.cc:260] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.327215 14185 consensus_queue.cc:260] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.327386 14186 ts_tablet_manager.cc:1428] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:14.327397 14184 ts_tablet_manager.cc:1428] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:14.327451 14185 ts_tablet_manager.cc:1428] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:38:14.327419 14204 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.327535 14204 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.327630 14190 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.327725 14190 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.328136 14192 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.328264 14192 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.592722 14190 raft_consensus.cc:482] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:14.592854 14190 raft_consensus.cc:504] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.593896 14190 leader_election.cc:288] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547)
I20260927 18:38:14.597478 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.594197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:55539 (local address 127.6.4.3:37547)
0927 18:38:14.594324 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.594329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.594525 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:38:14.594640 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.594644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.594683 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.594743 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.594973 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.595453 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.596578 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.596959 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.597048 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.597107 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.597296 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.597359 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:38:14.597514 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.594116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37547 (local address 127.6.4.2:55539)
0927 18:38:14.594452 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.594463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.594551 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.594760 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.594763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.594777 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.594884 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.594888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.595548 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.595551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.596473 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.596481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.597058 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.597061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.597186 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.597204 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.597266 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.597346 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":53,"thread_start_us":59,"threads_started":1}
I20260927 18:38:14.597513 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.594167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:49977 (local address 127.6.4.1:36125)
0927 18:38:14.594306 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.594311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.594453 (+ 142us) server_negotiation.cc:407] Connection header received
0927 18:38:14.594581 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.594583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.594621 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.594680 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:14.594955 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.595441 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.596602 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.596978 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.597064 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.597118 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.597329 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.597396 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 18:38:14.597534 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.594092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36125 (local address 127.6.4.2:49977)
0927 18:38:14.594377 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.594388 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.594488 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.594700 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.594703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.594724 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:14.594839 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.594845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.595538 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.595541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.596490 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.596499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.597075 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.597079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.597222 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.597239 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.597301 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.597389 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":138,"thread_start_us":81,"threads_started":1}
I20260927 18:38:14.597887 13830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a165bfcad1c0487bac061d68b1ad8a4c" candidate_uuid: "eb24b63588ca44898128ca98bb15083b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7" is_pre_election: true
I20260927 18:38:14.598016 13830 raft_consensus.cc:2454] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb24b63588ca44898128ca98bb15083b in term 0.
I20260927 18:38:14.598074 14110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a165bfcad1c0487bac061d68b1ad8a4c" candidate_uuid: "eb24b63588ca44898128ca98bb15083b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" is_pre_election: true
I20260927 18:38:14.598299 14110 raft_consensus.cc:2454] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb24b63588ca44898128ca98bb15083b in term 0.
I20260927 18:38:14.598335 13901 leader_election.cc:302] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302a820be6374d8d87174c73a2ac5be7, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:14.598567 14190 raft_consensus.cc:2781] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:14.598616 14190 raft_consensus.cc:482] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:14.598645 14190 raft_consensus.cc:3037] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.599572 14190 raft_consensus.cc:504] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.599874 14190 leader_election.cc:288] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [CANDIDATE]: Term 1 election: Requested vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547)
I20260927 18:38:14.600162 13830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a165bfcad1c0487bac061d68b1ad8a4c" candidate_uuid: "eb24b63588ca44898128ca98bb15083b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7"
I20260927 18:38:14.600204 14110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a165bfcad1c0487bac061d68b1ad8a4c" candidate_uuid: "eb24b63588ca44898128ca98bb15083b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9bc61d47431403ab7b74596cc2ba3c5"
I20260927 18:38:14.600283 13830 raft_consensus.cc:3037] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.600283 14110 raft_consensus.cc:3037] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:14.601286 13830 raft_consensus.cc:2454] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb24b63588ca44898128ca98bb15083b in term 1.
I20260927 18:38:14.601286 14110 raft_consensus.cc:2454] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb24b63588ca44898128ca98bb15083b in term 1.
I20260927 18:38:14.601541 13901 leader_election.cc:302] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302a820be6374d8d87174c73a2ac5be7, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:14.601686 14190 raft_consensus.cc:2781] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:14.601821 14190 raft_consensus.cc:686] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [term 1 LEADER]: Becoming Leader. State: Replica: eb24b63588ca44898128ca98bb15083b, State: Running, Role: LEADER
I20260927 18:38:14.602169 14190 consensus_queue.cc:237] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.603188 14225 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: TxnStatusTablet state changed. Reason: New leader eb24b63588ca44898128ca98bb15083b. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.603271 14225 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:14.603497 14227 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:38:14.603623 14227 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:38:14.604473 13671 catalog_manager.cc:5495] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b reported cstate change: term changed from 0 to 1, leader changed from <none> to eb24b63588ca44898128ca98bb15083b (127.6.4.2). New cstate: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:14.620303 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.616970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.6.4.2:35631)
0927 18:38:14.617184 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.617192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.617204 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:14.617386 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.617389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.617444 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.617532 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:14.617750 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.618252 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.619023 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.619244 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.619324 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.619577 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.619935 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.620028 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.620121 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.620165 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260927 18:38:14.620317 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.616922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:35631 (local address 127.0.0.1:60502)
0927 18:38:14.617152 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.617166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.617260 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.617553 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.617556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.617562 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:14.617653 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.617658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.618345 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.618348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.618924 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.618932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.619324 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.619338 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.619422 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.619460 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.620019 (+ 559us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.620033 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.620119 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.620200 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260927 18:38:14.622499 13830 raft_consensus.cc:1261] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Refusing update from remote peer eb24b63588ca44898128ca98bb15083b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:14.622684 14110 raft_consensus.cc:1261] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Refusing update from remote peer eb24b63588ca44898128ca98bb15083b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:14.622962 14225 consensus_queue.cc:1035] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [LEADER]: Connected to new peer: Peer: permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:14.623109 14190 consensus_queue.cc:1035] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:14.624431 14192 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: TxnStatusTablet state changed. Reason: New leader eb24b63588ca44898128ca98bb15083b. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.624555 14192 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.624644 14204 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: TxnStatusTablet state changed. Reason: New leader eb24b63588ca44898128ca98bb15083b. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.624729 14204 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.626392 14190 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.626485 14190 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:14.626557 14199 mvcc.cc:205] Tried to move back new op lower bound from 7334028470769299456 to 7334028470692356096. Current Snapshot: MvccSnapshot[applied={T|T < 7334028470769299456}]
I20260927 18:38:14.627593 14190 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.627684 14190 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P eb24b63588ca44898128ca98bb15083b: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:38:14.628015 14204 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.628129 14204 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P a9bc61d47431403ab7b74596cc2ba3c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.630985 14192 tablet_replica.cc:439] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb24b63588ca44898128ca98bb15083b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } } }
I20260927 18:38:14.631086 14192 tablet_replica.cc:442] T a165bfcad1c0487bac061d68b1ad8a4c P 302a820be6374d8d87174c73a2ac5be7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:38:14.641377 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.638423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:56245 (local address 127.6.4.2:35631)
0927 18:38:14.638522 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.638526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.638749 (+ 223us) server_negotiation.cc:407] Connection header received
0927 18:38:14.638827 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.638830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.638870 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.638929 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:14.639132 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.639603 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.640466 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.640677 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.640766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.641014 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.641073 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.641125 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.641225 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.641267 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:38:14.641409 14235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.638349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:35631 (local address 127.6.4.3:56245)
0927 18:38:14.638645 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.638656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.638751 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.638943 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.638946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.638953 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:14.639041 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.639047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.639696 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.639699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.640365 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.640374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.640763 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.640777 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.640860 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.640899 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.641140 (+ 241us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.641146 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.641212 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.641291 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":68,"threads_started":1}
I20260927 18:38:14.647084 13830 raft_consensus.cc:1261] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:14.647256 13970 raft_consensus.cc:1261] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:14.647552 14204 consensus_queue.cc:1035] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:14.647677 14194 consensus_queue.cc:1035] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:14.652150 14197 mvcc.cc:205] Tried to move back new op lower bound from 7334028470871584768 to 7334028469340520448. Current Snapshot: MvccSnapshot[applied={T|T < 7334028470871584768}]
I20260927 18:38:14.654568 14198 mvcc.cc:205] Tried to move back new op lower bound from 7334028470871584768 to 7334028469340520448. Current Snapshot: MvccSnapshot[applied={T|T < 7334028470871584768}]
I20260927 18:38:14.666441 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.662946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37547 (local address 127.6.4.2:58081)
0927 18:38:14.663242 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:14.663254 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:14.663335 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:14.663672 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:14.663675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:14.663690 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:38:14.663828 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.663834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.664665 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.664668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.665339 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:14.665348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.665735 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.665748 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:14.665839 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.665878 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.666147 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.666153 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:14.666231 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:38:14.666303 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":54,"threads_started":1}
I20260927 18:38:14.666414 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.662994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:58081 (local address 127.6.4.3:37547)
0927 18:38:14.663089 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.663093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.663353 (+ 260us) server_negotiation.cc:407] Connection header received
0927 18:38:14.663401 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.663403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.663450 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.663542 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:14.663931 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.664578 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.665444 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.665657 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.665738 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.665987 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:14.666053 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:14.666110 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.666241 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.666294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:38:14.683553 13673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:14.683701 13673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:14.685559 13673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Sending DeleteTablet for 3 replicas of tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0
I20260927 18:38:14.686033 14090 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:14 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:37284
I20260927 18:38:14.686149 13950 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:14 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60488
I20260927 18:38:14.686385 13810 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:14 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:59612
I20260927 18:38:14.686620 14245 tablet_replica.cc:330] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: stopping tablet replica
I20260927 18:38:14.686643 14244 tablet_replica.cc:330] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: stopping tablet replica
I20260927 18:38:14.686743 14244 raft_consensus.cc:2229] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:14.686849 14244 raft_consensus.cc:2258] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:14.686769 14245 raft_consensus.cc:2229] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:14.687091 14245 raft_consensus.cc:2258] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:14.687449 14244 ts_tablet_manager.cc:1905] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:14.687738 14245 ts_tablet_manager.cc:1905] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:14.690361 14244 ts_tablet_manager.cc:1918] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:14.690438 14244 log.cc:1196] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/0c1027f6472f4a8f9e47f0fed1d5d1d0
I20260927 18:38:14.690382 14245 ts_tablet_manager.cc:1918] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:14.690593 14245 log.cc:1196] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/0c1027f6472f4a8f9e47f0fed1d5d1d0
I20260927 18:38:14.690666 14244 ts_tablet_manager.cc:1939] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P eb24b63588ca44898128ca98bb15083b: Deleting consensus metadata
I20260927 18:38:14.690903 14245 ts_tablet_manager.cc:1939] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P 302a820be6374d8d87174c73a2ac5be7: Deleting consensus metadata
I20260927 18:38:14.690974 14243 tablet_replica.cc:330] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: stopping tablet replica
I20260927 18:38:14.691087 14243 raft_consensus.cc:2229] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:14.691237 14243 raft_consensus.cc:2258] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:14.691241 13660 catalog_manager.cc:4841] TS eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631): tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 (table test-table [id=68f90c58df704856ac2380b0b2e989bf]) successfully deleted
I20260927 18:38:14.691753 14243 ts_tablet_manager.cc:1905] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:14.693186 13657 catalog_manager.cc:4841] TS 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125): tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 (table test-table [id=68f90c58df704856ac2380b0b2e989bf]) successfully deleted
I20260927 18:38:14.694343 14243 ts_tablet_manager.cc:1918] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:14.694416 14243 log.cc:1196] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/0c1027f6472f4a8f9e47f0fed1d5d1d0
I20260927 18:38:14.694650 14243 ts_tablet_manager.cc:1939] T 0c1027f6472f4a8f9e47f0fed1d5d1d0 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting consensus metadata
I20260927 18:38:14.695175 13658 catalog_manager.cc:4841] TS a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547): tablet 0c1027f6472f4a8f9e47f0fed1d5d1d0 (table test-table [id=68f90c58df704856ac2380b0b2e989bf]) successfully deleted
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:38:14.715394 13722 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:14.709316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36220 (local address 127.6.4.62:37971)
0927 18:38:14.709417 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.709422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.709621 (+ 199us) server_negotiation.cc:407] Connection header received
0927 18:38:14.709734 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.709737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.709780 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.709857 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:14.710528 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.711068 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.711861 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.712082 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.712195 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.715281 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36220: BlockingRecv error: recv got EOF from 127.0.0.1:36220 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 27 18:38:14 dist-test-slave-bzzj krb5kdc[13628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM
I20260927 18:38:14.723945 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:14.716811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36236 (local address 127.6.4.62:37971)
0927 18:38:14.716912 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:14.716916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:14.717050 (+ 134us) server_negotiation.cc:407] Connection header received
0927 18:38:14.717145 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:14.717147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:14.717185 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:14.717242 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:14.717764 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.718239 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.718953 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:14.719154 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:14.719233 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:14.721225 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:14.721228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:14.721230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:14.721251 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:14.722536 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.722920 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.722924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.722926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.722996 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:14.723224 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:14.723227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:14.723229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:14.723393 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:14.723500 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:14.723706 (+ 206us) server_negotiation.cc:299] Negotiation successful
0927 18:38:14.723806 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:38:14.726207 13673 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:14.726600 13673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:14.730880 13810 tablet_service.cc:1467] Processing CreateTablet for tablet c4d45efd63fe4000bf1fa1b8099f4432 (DEFAULT_TABLE table=test-table [id=c7f7ed36394143809dd82660763ed4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.730877 13950 tablet_service.cc:1467] Processing CreateTablet for tablet c4d45efd63fe4000bf1fa1b8099f4432 (DEFAULT_TABLE table=test-table [id=c7f7ed36394143809dd82660763ed4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.731130 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d45efd63fe4000bf1fa1b8099f4432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.731130 13950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d45efd63fe4000bf1fa1b8099f4432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.731099 14090 tablet_service.cc:1467] Processing CreateTablet for tablet c4d45efd63fe4000bf1fa1b8099f4432 (DEFAULT_TABLE table=test-table [id=c7f7ed36394143809dd82660763ed4f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:14.731376 14090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d45efd63fe4000bf1fa1b8099f4432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:14.733363 14184 tablet_bootstrap.cc:492] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Bootstrap starting.
I20260927 18:38:14.733834 14185 tablet_bootstrap.cc:492] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Bootstrap starting.
I20260927 18:38:14.734149 14184 tablet_bootstrap.cc:654] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.734670 14186 tablet_bootstrap.cc:492] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Bootstrap starting.
I20260927 18:38:14.734673 14185 tablet_bootstrap.cc:654] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.735432 14186 tablet_bootstrap.cc:654] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:14.736047 14185 tablet_bootstrap.cc:492] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: No bootstrap required, opened a new log
I20260927 18:38:14.736157 14185 ts_tablet_manager.cc:1397] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260927 18:38:14.736433 14186 tablet_bootstrap.cc:492] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: No bootstrap required, opened a new log
I20260927 18:38:14.736505 14186 ts_tablet_manager.cc:1397] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:14.736569 14185 raft_consensus.cc:348] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.736688 14185 raft_consensus.cc:374] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.736729 14185 raft_consensus.cc:729] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.736852 14185 consensus_queue.cc:260] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.737015 14186 raft_consensus.cc:348] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.737115 14186 raft_consensus.cc:374] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.737118 14185 ts_tablet_manager.cc:1428] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:38:14.737151 14186 raft_consensus.cc:729] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302a820be6374d8d87174c73a2ac5be7, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.737270 14186 consensus_queue.cc:260] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.737489 14186 ts_tablet_manager.cc:1428] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:14.738236 14184 tablet_bootstrap.cc:492] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: No bootstrap required, opened a new log
I20260927 18:38:14.738315 14184 ts_tablet_manager.cc:1397] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260927 18:38:14.738713 14184 raft_consensus.cc:348] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.738806 14184 raft_consensus.cc:374] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:14.738838 14184 raft_consensus.cc:729] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb24b63588ca44898128ca98bb15083b, State: Initialized, Role: FOLLOWER
I20260927 18:38:14.738950 14184 consensus_queue.cc:260] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:14.739168 14184 ts_tablet_manager.cc:1428] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:15.075778 14241 raft_consensus.cc:482] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:15.075918 14241 raft_consensus.cc:504] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.076344 14241 leader_election.cc:288] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:15.076807 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d45efd63fe4000bf1fa1b8099f4432" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b" is_pre_election: true
I20260927 18:38:15.076828 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d45efd63fe4000bf1fa1b8099f4432" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7" is_pre_election: true
I20260927 18:38:15.076948 13970 raft_consensus.cc:2454] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:15.076978 13829 raft_consensus.cc:2454] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:15.077251 14044 leader_election.cc:302] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bc61d47431403ab7b74596cc2ba3c5, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:15.077419 14241 raft_consensus.cc:2781] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:15.077478 14241 raft_consensus.cc:482] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:15.077504 14241 raft_consensus.cc:3037] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:15.078454 14241 raft_consensus.cc:504] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.078732 14241 leader_election.cc:288] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Requested vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:15.079085 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d45efd63fe4000bf1fa1b8099f4432" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7"
I20260927 18:38:15.079094 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d45efd63fe4000bf1fa1b8099f4432" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b"
I20260927 18:38:15.079181 13829 raft_consensus.cc:3037] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:15.079191 13970 raft_consensus.cc:3037] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:15.080183 13829 raft_consensus.cc:2454] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:15.080183 13970 raft_consensus.cc:2454] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:15.080408 14041 leader_election.cc:302] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302a820be6374d8d87174c73a2ac5be7, a9bc61d47431403ab7b74596cc2ba3c5; no voters:
I20260927 18:38:15.080602 14241 raft_consensus.cc:2781] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:15.080686 14241 raft_consensus.cc:686] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Becoming Leader. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Running, Role: LEADER
I20260927 18:38:15.080814 14241 consensus_queue.cc:237] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.082324 13673 catalog_manager.cc:5495] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260927 18:38:15.101570 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:15.098916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.6.4.3:37547)
0927 18:38:15.099026 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:15.099029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:15.099042 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:15.099117 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:15.099119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:15.099163 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:15.099225 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:15.099400 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:15.099880 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:15.100780 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:15.100989 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:15.101063 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:15.101240 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:15.101312 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:15.101357 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:15.101414 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 18:38:15.101451 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:38:15.103336 13829 raft_consensus.cc:1261] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:15.103474 13970 raft_consensus.cc:1261] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:15.103693 14204 consensus_queue.cc:1035] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:15.103819 14241 consensus_queue.cc:1035] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:15.109511 13671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:15.109642 13671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:15.111351 13671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Sending DeleteTablet for 3 replicas of tablet c4d45efd63fe4000bf1fa1b8099f4432
I20260927 18:38:15.111842 13950 tablet_service.cc:1514] Processing DeleteTablet for tablet c4d45efd63fe4000bf1fa1b8099f4432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:15 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60488
I20260927 18:38:15.111871 14090 tablet_service.cc:1514] Processing DeleteTablet for tablet c4d45efd63fe4000bf1fa1b8099f4432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:15 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:37284
I20260927 18:38:15.111855 13810 tablet_service.cc:1514] Processing DeleteTablet for tablet c4d45efd63fe4000bf1fa1b8099f4432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:15 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:59612
I20260927 18:38:15.111996 14244 tablet_replica.cc:330] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: stopping tablet replica
I20260927 18:38:15.112013 14243 tablet_replica.cc:330] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: stopping tablet replica
I20260927 18:38:15.112113 14245 tablet_replica.cc:330] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: stopping tablet replica
I20260927 18:38:15.112128 14243 raft_consensus.cc:2229] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:15.112142 14244 raft_consensus.cc:2229] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:15.112212 14245 raft_consensus.cc:2229] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:15.112253 14244 raft_consensus.cc:2258] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:15.112278 14243 raft_consensus.cc:2258] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:15.112321 14245 raft_consensus.cc:2258] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:15.112895 14245 ts_tablet_manager.cc:1905] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:15.112895 14243 ts_tablet_manager.cc:1905] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:15.113195 13671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:38:15.113307 14244 ts_tablet_manager.cc:1905] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:38:15.113550 13671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:15.115135 14245 ts_tablet_manager.cc:1918] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:15.115135 14243 ts_tablet_manager.cc:1918] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:15.115196 14245 log.cc:1196] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/c4d45efd63fe4000bf1fa1b8099f4432
I20260927 18:38:15.115195 14243 log.cc:1196] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/c4d45efd63fe4000bf1fa1b8099f4432
I20260927 18:38:15.115422 14243 ts_tablet_manager.cc:1939] T c4d45efd63fe4000bf1fa1b8099f4432 P a9bc61d47431403ab7b74596cc2ba3c5: Deleting consensus metadata
I20260927 18:38:15.115422 14245 ts_tablet_manager.cc:1939] T c4d45efd63fe4000bf1fa1b8099f4432 P 302a820be6374d8d87174c73a2ac5be7: Deleting consensus metadata
I20260927 18:38:15.115857 14244 ts_tablet_manager.cc:1918] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:15.115932 14244 log.cc:1196] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/c4d45efd63fe4000bf1fa1b8099f4432
I20260927 18:38:15.116197 14244 ts_tablet_manager.cc:1939] T c4d45efd63fe4000bf1fa1b8099f4432 P eb24b63588ca44898128ca98bb15083b: Deleting consensus metadata
I20260927 18:38:15.116348 13658 catalog_manager.cc:4841] TS a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547): tablet c4d45efd63fe4000bf1fa1b8099f4432 (table test-table [id=c7f7ed36394143809dd82660763ed4f5]) successfully deleted
I20260927 18:38:15.116806 13660 catalog_manager.cc:4841] TS eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631): tablet c4d45efd63fe4000bf1fa1b8099f4432 (table test-table [id=c7f7ed36394143809dd82660763ed4f5]) successfully deleted
I20260927 18:38:15.117161 13657 catalog_manager.cc:4841] TS 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125): tablet c4d45efd63fe4000bf1fa1b8099f4432 (table test-table [id=c7f7ed36394143809dd82660763ed4f5]) successfully deleted
I20260927 18:38:15.118045 13810 tablet_service.cc:1467] Processing CreateTablet for tablet 485a6da1a7a44b7987b733624886f34c (DEFAULT_TABLE table=test-table [id=6383f0ba5b394654bca6280e79cd9c7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:15.118160 13950 tablet_service.cc:1467] Processing CreateTablet for tablet 485a6da1a7a44b7987b733624886f34c (DEFAULT_TABLE table=test-table [id=6383f0ba5b394654bca6280e79cd9c7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:15.118273 13810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 485a6da1a7a44b7987b733624886f34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:15.118237 14090 tablet_service.cc:1467] Processing CreateTablet for tablet 485a6da1a7a44b7987b733624886f34c (DEFAULT_TABLE table=test-table [id=6383f0ba5b394654bca6280e79cd9c7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:15.118377 13950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 485a6da1a7a44b7987b733624886f34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:15.118435 14090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 485a6da1a7a44b7987b733624886f34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:15.120172 14184 tablet_bootstrap.cc:492] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Bootstrap starting.
I20260927 18:38:15.120478 14185 tablet_bootstrap.cc:492] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Bootstrap starting.
I20260927 18:38:15.120859 14184 tablet_bootstrap.cc:654] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:15.121035 14185 tablet_bootstrap.cc:654] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:15.121335 14186 tablet_bootstrap.cc:492] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: Bootstrap starting.
I20260927 18:38:15.121928 14185 tablet_bootstrap.cc:492] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: No bootstrap required, opened a new log
I20260927 18:38:15.121980 14185 ts_tablet_manager.cc:1397] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:38:15.122020 14186 tablet_bootstrap.cc:654] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:15.122248 14185 raft_consensus.cc:348] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.122335 14185 raft_consensus.cc:374] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:15.122356 14185 raft_consensus.cc:729] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Initialized, Role: FOLLOWER
I20260927 18:38:15.122447 14185 consensus_queue.cc:260] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.122681 14185 ts_tablet_manager.cc:1428] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:38:15.122808 14184 tablet_bootstrap.cc:492] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: No bootstrap required, opened a new log
I20260927 18:38:15.122874 14184 ts_tablet_manager.cc:1397] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:38:15.123155 14184 raft_consensus.cc:348] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.123221 14184 raft_consensus.cc:374] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:15.123240 14184 raft_consensus.cc:729] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb24b63588ca44898128ca98bb15083b, State: Initialized, Role: FOLLOWER
I20260927 18:38:15.123337 14184 consensus_queue.cc:260] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.123543 14184 ts_tablet_manager.cc:1428] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:38:15.124353 14186 tablet_bootstrap.cc:492] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: No bootstrap required, opened a new log
I20260927 18:38:15.124446 14186 ts_tablet_manager.cc:1397] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:38:15.124826 14186 raft_consensus.cc:348] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.124931 14186 raft_consensus.cc:374] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:15.124969 14186 raft_consensus.cc:729] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302a820be6374d8d87174c73a2ac5be7, State: Initialized, Role: FOLLOWER
I20260927 18:38:15.125077 14186 consensus_queue.cc:260] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.125325 14186 ts_tablet_manager.cc:1428] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:38:15.127664 14204 raft_consensus.cc:482] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:15.127733 14204 raft_consensus.cc:504] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.127997 14204 leader_election.cc:288] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:15.128316 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485a6da1a7a44b7987b733624886f34c" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b" is_pre_election: true
I20260927 18:38:15.128320 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485a6da1a7a44b7987b733624886f34c" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7" is_pre_election: true
I20260927 18:38:15.128427 13970 raft_consensus.cc:2454] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:15.128427 13829 raft_consensus.cc:2454] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 0.
I20260927 18:38:15.128695 14041 leader_election.cc:302] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302a820be6374d8d87174c73a2ac5be7, a9bc61d47431403ab7b74596cc2ba3c5; no voters:
I20260927 18:38:15.128850 14204 raft_consensus.cc:2781] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:15.128903 14204 raft_consensus.cc:482] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:15.128929 14204 raft_consensus.cc:3037] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:15.129604 14204 raft_consensus.cc:504] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.129877 14204 leader_election.cc:288] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Requested vote from peers 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125), eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631)
I20260927 18:38:15.130133 13829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485a6da1a7a44b7987b733624886f34c" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302a820be6374d8d87174c73a2ac5be7"
I20260927 18:38:15.130147 13970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "485a6da1a7a44b7987b733624886f34c" candidate_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb24b63588ca44898128ca98bb15083b"
I20260927 18:38:15.130235 13970 raft_consensus.cc:3037] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:15.130236 13829 raft_consensus.cc:3037] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:15.131119 13970 raft_consensus.cc:2454] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:15.131119 13829 raft_consensus.cc:2454] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bc61d47431403ab7b74596cc2ba3c5 in term 1.
I20260927 18:38:15.131357 14044 leader_election.cc:302] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bc61d47431403ab7b74596cc2ba3c5, eb24b63588ca44898128ca98bb15083b; no voters:
I20260927 18:38:15.131501 14204 raft_consensus.cc:2781] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:15.131582 14204 raft_consensus.cc:686] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Becoming Leader. State: Replica: a9bc61d47431403ab7b74596cc2ba3c5, State: Running, Role: LEADER
I20260927 18:38:15.131704 14204 consensus_queue.cc:237] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } }
I20260927 18:38:15.133006 13673 catalog_manager.cc:5495] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9bc61d47431403ab7b74596cc2ba3c5" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260927 18:38:15.147351 13829 raft_consensus.cc:1261] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:15.147634 13970 raft_consensus.cc:1261] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Refusing update from remote peer a9bc61d47431403ab7b74596cc2ba3c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:15.147688 14204 consensus_queue.cc:1035] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "302a820be6374d8d87174c73a2ac5be7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:15.147989 14204 consensus_queue.cc:1035] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb24b63588ca44898128ca98bb15083b" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 35631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:15.171262 13671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:15.171386 13671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:15.173028 13671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a82e6ca9c1e425e93cbb74f77212512: Sending DeleteTablet for 3 replicas of tablet 485a6da1a7a44b7987b733624886f34c
I20260927 18:38:15.173468 13810 tablet_service.cc:1514] Processing DeleteTablet for tablet 485a6da1a7a44b7987b733624886f34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:15 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:59612
I20260927 18:38:15.173514 13950 tablet_service.cc:1514] Processing DeleteTablet for tablet 485a6da1a7a44b7987b733624886f34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:15 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60488
I20260927 18:38:15.173547 14090 tablet_service.cc:1514] Processing DeleteTablet for tablet 485a6da1a7a44b7987b733624886f34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:15 UTC) from {username='oryx', principal='oryx/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:37284
I20260927 18:38:15.173645 14245 tablet_replica.cc:330] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: stopping tablet replica
I20260927 18:38:15.173691 14243 tablet_replica.cc:330] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: stopping tablet replica
I20260927 18:38:15.173746 14245 raft_consensus.cc:2229] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:15.173786 14243 raft_consensus.cc:2229] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:15.173825 14245 raft_consensus.cc:2258] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:15.173928 14243 raft_consensus.cc:2258] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:15.173957 14244 tablet_replica.cc:330] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: stopping tablet replica
I20260927 18:38:15.174062 14244 raft_consensus.cc:2229] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:15.174157 14244 raft_consensus.cc:2258] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:15.174536 14243 ts_tablet_manager.cc:1905] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:15.174567 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 13745
I20260927 18:38:15.174917 14245 ts_tablet_manager.cc:1905] T 485a6da1a7a44b7987b733624886f34c P 302a820be6374d8d87174c73a2ac5be7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:15.176745 14243 ts_tablet_manager.cc:1918] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:15.176820 14243 log.cc:1196] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/485a6da1a7a44b7987b733624886f34c
I20260927 18:38:15.176884 14244 ts_tablet_manager.cc:1905] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:15.177042 14243 ts_tablet_manager.cc:1939] T 485a6da1a7a44b7987b733624886f34c P a9bc61d47431403ab7b74596cc2ba3c5: Deleting consensus metadata
I20260927 18:38:15.177616 13658 catalog_manager.cc:4841] TS a9bc61d47431403ab7b74596cc2ba3c5 (127.6.4.3:37547): tablet 485a6da1a7a44b7987b733624886f34c (table test-table [id=6383f0ba5b394654bca6280e79cd9c7b]) successfully deleted
I20260927 18:38:15.179054 14244 ts_tablet_manager.cc:1918] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:38:15.179131 14244 log.cc:1196] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/485a6da1a7a44b7987b733624886f34c
I20260927 18:38:15.179344 14244 ts_tablet_manager.cc:1939] T 485a6da1a7a44b7987b733624886f34c P eb24b63588ca44898128ca98bb15083b: Deleting consensus metadata
I20260927 18:38:15.179886 13660 catalog_manager.cc:4841] TS eb24b63588ca44898128ca98bb15083b (127.6.4.2:35631): tablet 485a6da1a7a44b7987b733624886f34c (table test-table [id=6383f0ba5b394654bca6280e79cd9c7b]) successfully deleted
W20260927 18:38:15.182646 13657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:36125 (error 108)
I20260927 18:38:15.182708 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 13885
I20260927 18:38:15.183981 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:15.183640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36125 (local address 127.0.0.1:59620)
0927 18:38:15.183847 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:15.183907 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:36125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":51,"threads_started":1}
W20260927 18:38:15.184116 13657 catalog_manager.cc:4584] TS 302a820be6374d8d87174c73a2ac5be7 (127.6.4.1:36125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 485a6da1a7a44b7987b733624886f34c: Network error: Client connection negotiation failed: client connection to 127.6.4.1:36125: connect: Connection refused (error 111)
I20260927 18:38:15.189435 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14025
I20260927 18:38:15.194411 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 13641
2026-09-27T18:38:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3084 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:15 dist-test-slave-bzzj krb5kdc[14275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:15 dist-test-slave-bzzj krb5kdc[14275](info): set up 2 sockets
Sep 27 18:38:15 dist-test-slave-bzzj krb5kdc[14275](info): commencing operation
krb5kdc: starting...
W20260927 18:38:16.148447 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:16 dist-test-slave-bzzj krb5kdc[14275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:38:16.256943 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40383
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:16.359406 14288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:16.359619 14288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:16.359658 14288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:16.363523 14288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:16.363569 14288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:16.363586 14288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:16.363601 14288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:16.363623 14288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:16.368564 14288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:16.369427 14288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:16.370297 14288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:16.379303 14293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:16.379312 14296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:16.379407 14288 server_base.cc:1034] running on GCE node
W20260927 18:38:16.379303 14294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:16.380138 14288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:16.381137 14288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:16.382290 14288 hybrid_clock.cc:648] HybridClock initialized: now 1790534296382273 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T18:38:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1200 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:16 dist-test-slave-bzzj krb5kdc[14301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:16 dist-test-slave-bzzj krb5kdc[14301](info): set up 2 sockets
Sep 27 18:38:16 dist-test-slave-bzzj krb5kdc[14301](info): commencing operation
krb5kdc: starting...
W20260927 18:38:17.332258 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:17 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:38:17.436636 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42869
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:17.537732 14314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:17.537952 14314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:17.537987 14314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:17.541829 14314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:17.541873 14314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:17.541890 14314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:17.541904 14314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:17.541918 14314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:17.546876 14314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:17.547746 14314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:17.548683 14314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:17.556262 14322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:17.556320 14314 server_base.cc:1034] running on GCE node
W20260927 18:38:17.556275 14319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:17.556278 14320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:17.557022 14314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:17.557951 14314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:17.559108 14314 hybrid_clock.cc:648] HybridClock initialized: now 1790534297559092 us; error 27 us; skew 500 ppm
Sep 27 18:38:17 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534297, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:17.565632 14314 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:38:17.566380 14314 webserver.cc:460] Webserver started at http://127.6.4.62:33075/ using document root <none> and password file <none>
I20260927 18:38:17.566844 14314 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:17.566896 14314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:17.567044 14314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:17.568898 14314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "635bf104307a4eb885778a2cd158ca20"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "47cadab63000a62e456b69a8e67f78b2"
server_key_iv: "dff2764233ef029dc8ba6bbf6f31a03c"
server_key_version: "encryptionkey@0"
I20260927 18:38:17.569378 14314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "635bf104307a4eb885778a2cd158ca20"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "47cadab63000a62e456b69a8e67f78b2"
server_key_iv: "dff2764233ef029dc8ba6bbf6f31a03c"
server_key_version: "encryptionkey@0"
I20260927 18:38:17.572629 14314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:38:17.574736 14329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:17.575814 14314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:17.575904 14314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "635bf104307a4eb885778a2cd158ca20"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "47cadab63000a62e456b69a8e67f78b2"
server_key_iv: "dff2764233ef029dc8ba6bbf6f31a03c"
server_key_version: "encryptionkey@0"
I20260927 18:38:17.575978 14314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:17.592607 14314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:17.595314 14314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:17.595453 14314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:17.601691 14314 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:42869
I20260927 18:38:17.601701 14381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:42869 every 8 connection(s)
I20260927 18:38:17.602593 14314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:17.605194 14382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:17.610296 14382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Bootstrap starting.
I20260927 18:38:17.611074 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14314
I20260927 18:38:17.611207 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:17.611491 6160 external_mini_cluster.cc:1428] Setting key 6de0f09c1a2a8c046f414382cc555298
I20260927 18:38:17.612998 14382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:17.613689 14382 log.cc:824] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:17.615654 14382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: No bootstrap required, opened a new log
I20260927 18:38:17.619575 14382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:17.619825 14382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:17.619864 14382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635bf104307a4eb885778a2cd158ca20, State: Initialized, Role: FOLLOWER
Sep 27 18:38:17 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:17.620299 14382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:17.620438 14382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:17.620503 14382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:17.620594 14382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:17.621555 14382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:17.621894 14382 leader_election.cc:302] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 635bf104307a4eb885778a2cd158ca20; no voters:
I20260927 18:38:17.622252 14382 leader_election.cc:288] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:17.622357 14387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:17.622538 14387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 LEADER]: Becoming Leader. State: Replica: 635bf104307a4eb885778a2cd158ca20, State: Running, Role: LEADER
I20260927 18:38:17.622853 14387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:17.623440 14382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:17.624465 14389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "635bf104307a4eb885778a2cd158ca20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } } }
I20260927 18:38:17.624578 14389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:17.624891 14388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635bf104307a4eb885778a2cd158ca20. Latest consensus state: current_term: 1 leader_uuid: "635bf104307a4eb885778a2cd158ca20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } } }
I20260927 18:38:17.624989 14388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:17.625097 14396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:17.626152 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:17.612555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.6.4.62:42869)
0927 18:38:17.612958 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:17.612972 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:17.613008 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:38:17.613662 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:17.613678 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:17.614038 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:17.614408 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:17.615267 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:17.616344 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:17.617553 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:17.618046 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:17.618169 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:17.620894 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:17.620900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:17.620917 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:17.620944 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:17.622770 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:17.623246 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:17.623251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:17.623256 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:17.623392 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:17.623694 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:17.623697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:17.623699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:17.624191 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:17.624387 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:17.624961 (+ 574us) server_negotiation.cc:299] Negotiation successful
0927 18:38:17.625246 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":116,"threads_started":1}
I20260927 18:38:17.627810 14396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:17.632617 14396 catalog_manager.cc:1347] Generated new cluster ID: 5770177db609429fb66e26b974029525
I20260927 18:38:17.632661 14396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:17.654955 14396 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:17.655922 14396 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:17.660015 14396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Generated new TSK 0
I20260927 18:38:17.660650 14396 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:38:17.686982 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42869
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:17.789227 14408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:17.789443 14408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:17.789484 14408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:17.793363 14408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:17.793411 14408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:17.793462 14408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:17.798346 14408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:17.799175 14408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:17.800071 14408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:17.808871 14413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:17.808944 14408 server_base.cc:1034] running on GCE node
W20260927 18:38:17.808912 14414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:17.808876 14416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:17.809386 14408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:17.809940 14408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:17.811096 14408 hybrid_clock.cc:648] HybridClock initialized: now 1790534297811082 us; error 24 us; skew 500 ppm
Sep 27 18:38:17 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534297, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:17.813730 14408 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:38:17.814500 14408 webserver.cc:460] Webserver started at http://127.6.4.1:37175/ using document root <none> and password file <none>
I20260927 18:38:17.815014 14408 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:17.815088 14408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:17.815265 14408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:17.817167 14408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "7eed816b942a4e07ab5983da32c6d774"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "06ff5bb55e0b4af36fc35acce6e81c63"
server_key_iv: "a869f54c09acefd244ed30ce6f1ee5b2"
server_key_version: "encryptionkey@0"
I20260927 18:38:17.817656 14408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "7eed816b942a4e07ab5983da32c6d774"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "06ff5bb55e0b4af36fc35acce6e81c63"
server_key_iv: "a869f54c09acefd244ed30ce6f1ee5b2"
server_key_version: "encryptionkey@0"
I20260927 18:38:17.821076 14408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:38:17.823307 14423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:17.824568 14408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:17.824667 14408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "7eed816b942a4e07ab5983da32c6d774"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "06ff5bb55e0b4af36fc35acce6e81c63"
server_key_iv: "a869f54c09acefd244ed30ce6f1ee5b2"
server_key_version: "encryptionkey@0"
I20260927 18:38:17.824743 14408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:17.850608 14408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:17.853674 14408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:17.853818 14408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:17.854339 14408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:17.855093 14408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:17.855139 14408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:17.855178 14408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:17.855198 14408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:17.863535 14408 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:43833
I20260927 18:38:17.863579 14536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:43833 every 8 connection(s)
I20260927 18:38:17.864409 14408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:17.872180 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14408
I20260927 18:38:17.872295 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:17.872536 6160 external_mini_cluster.cc:1428] Setting key 2cd5719f742160d945e970e6ccc23649
Sep 27 18:38:17 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534297, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:17.876876 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:17.866093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:39265 (local address 127.6.4.62:42869)
0927 18:38:17.866235 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:17.866238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:17.866979 (+ 741us) server_negotiation.cc:407] Connection header received
0927 18:38:17.868234 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:17.868237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:17.868283 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:17.868357 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:17.869497 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:17.869998 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:17.871017 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:17.871239 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:17.871306 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:17.873589 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:17.873592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:17.873594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:17.873616 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:17.874971 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:17.875412 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:17.875415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:17.875417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:17.875504 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:17.875855 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:17.875857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:17.875858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:17.876012 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:17.876079 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:17.876624 (+ 545us) server_negotiation.cc:299] Negotiation successful
0927 18:38:17.876734 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 18:38:17.877656 14539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:17.866424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:39265)
0927 18:38:17.866840 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:17.866881 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:17.868018 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:17.868549 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:17.868556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:17.868937 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:17.869384 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:17.869397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:17.870085 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:17.870087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:17.870929 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:17.870937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:17.871304 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:17.871342 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:17.871467 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:17.871939 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:17.871954 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:17.873446 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:17.875068 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:17.875073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:17.875083 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:17.875319 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:17.875595 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:17.875598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:17.875599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:17.875775 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:17.876115 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:17.876122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:17.876398 (+ 276us) client_negotiation.cc:769] Sending connection context
0927 18:38:17.876598 (+ 200us) client_negotiation.cc:240] Negotiation successful
0927 18:38:17.876872 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":113,"threads_started":1}
I20260927 18:38:17.878863 14537 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42869
I20260927 18:38:17.879068 14537 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:17.879528 14537 heartbeater.cc:511] Master 127.6.4.62:42869 requested a full tablet report, sending...
I20260927 18:38:17.880966 14346 ts_manager.cc:194] Registered new tserver with Master: 7eed816b942a4e07ab5983da32c6d774 (127.6.4.1:43833)
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:38:17.882045 14346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:39265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:38:17.898309 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42869
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:18.001507 14542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:18.001752 14542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:18.001806 14542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:18.005858 14542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:18.005915 14542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:18.005987 14542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:18.011078 14542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:18.012030 14542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:18.013000 14542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:18.021901 14550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.022011 14542 server_base.cc:1034] running on GCE node
W20260927 18:38:18.021898 14548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.021898 14547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.022500 14542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:18.023074 14542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:18.024236 14542 hybrid_clock.cc:648] HybridClock initialized: now 1790534298024210 us; error 41 us; skew 500 ppm
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:18.026747 14542 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:38:18.027560 14542 webserver.cc:460] Webserver started at http://127.6.4.2:42849/ using document root <none> and password file <none>
I20260927 18:38:18.028049 14542 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:18.028136 14542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:18.028321 14542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:18.030161 14542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "98dc832659984a1cb661b3b5f45be3fc"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "67b8523f0f939e52ebf748bcd2a80f4d"
server_key_iv: "9dafdeb2ca6fc0939fc62b6dd11461f1"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.030629 14542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "98dc832659984a1cb661b3b5f45be3fc"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "67b8523f0f939e52ebf748bcd2a80f4d"
server_key_iv: "9dafdeb2ca6fc0939fc62b6dd11461f1"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.033950 14542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260927 18:38:18.036054 14557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.037202 14542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:18.037305 14542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "98dc832659984a1cb661b3b5f45be3fc"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "67b8523f0f939e52ebf748bcd2a80f4d"
server_key_iv: "9dafdeb2ca6fc0939fc62b6dd11461f1"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.037390 14542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:18.052295 14542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:18.054860 14542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:18.055011 14542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:18.055522 14542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:18.056270 14542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:18.056314 14542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.056365 14542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:18.056393 14542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.064188 14542 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:35547
I20260927 18:38:18.064240 14670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:35547 every 8 connection(s)
I20260927 18:38:18.064986 14542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:18.073441 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14542
I20260927 18:38:18.073540 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:18.073832 6160 external_mini_cluster.cc:1428] Setting key 4d92781525b9b478c1dd6296f8822567
I20260927 18:38:18.077450 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.066603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:38547 (local address 127.6.4.62:42869)
0927 18:38:18.066728 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.066732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.067511 (+ 779us) server_negotiation.cc:407] Connection header received
0927 18:38:18.068765 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.068769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.068823 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.068897 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.069944 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.070527 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.071602 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.071815 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.071901 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.074223 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:18.074227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:18.074229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:18.074277 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:18.075639 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:18.076069 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:18.076073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:18.076074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:18.076175 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:18.076462 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:18.076465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:18.076468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:18.076628 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:18.076707 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:18.077182 (+ 475us) server_negotiation.cc:299] Negotiation successful
0927 18:38:18.077288 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:38:18.078171 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.066985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:38547)
0927 18:38:18.067383 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.067420 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.068570 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.069059 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.069065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.069377 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.069803 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.069815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.070645 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.070648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.071490 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.071498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.071896 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.071946 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.072124 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.072604 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.072619 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.074064 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:18.075735 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:18.075740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:18.075754 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:18.075983 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:18.076253 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:18.076256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:18.076258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:18.076381 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:18.076718 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:18.076725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:18.076956 (+ 231us) client_negotiation.cc:769] Sending connection context
0927 18:38:18.077122 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 18:38:18.077417 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":113,"threads_started":1}
I20260927 18:38:18.079223 14671 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42869
I20260927 18:38:18.079433 14671 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:18.079850 14671 heartbeater.cc:511] Master 127.6.4.62:42869 requested a full tablet report, sending...
I20260927 18:38:18.080919 14346 ts_manager.cc:194] Registered new tserver with Master: 98dc832659984a1cb661b3b5f45be3fc (127.6.4.2:35547)
I20260927 18:38:18.081653 14346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:38547
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:38:18.098361 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42869
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:18.202337 14676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:18.202584 14676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:18.202621 14676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:18.206530 14676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:18.206578 14676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:18.206634 14676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:18.211557 14676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:18.212466 14676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:18.213392 14676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:18.222237 14681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.222316 14676 server_base.cc:1034] running on GCE node
W20260927 18:38:18.222234 14682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.222306 14684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.222829 14676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:18.223366 14676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:18.224535 14676 hybrid_clock.cc:648] HybridClock initialized: now 1790534298224521 us; error 27 us; skew 500 ppm
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:18.226991 14676 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:38:18.227810 14676 webserver.cc:460] Webserver started at http://127.6.4.3:33895/ using document root <none> and password file <none>
I20260927 18:38:18.228343 14676 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:18.228405 14676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:18.228587 14676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:18.230460 14676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "879db6a1bf3243cfb1ae62d23fe3f0ee"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "f7a095a8f4f374133b7be4f17e25a09a"
server_key_iv: "582cc3cc5ccb6f84945f6bbdca6cadbf"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.230924 14676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "879db6a1bf3243cfb1ae62d23fe3f0ee"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "f7a095a8f4f374133b7be4f17e25a09a"
server_key_iv: "582cc3cc5ccb6f84945f6bbdca6cadbf"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.234278 14676 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:18.236383 14691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.237536 14676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:38:18.237686 14676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "879db6a1bf3243cfb1ae62d23fe3f0ee"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "f7a095a8f4f374133b7be4f17e25a09a"
server_key_iv: "582cc3cc5ccb6f84945f6bbdca6cadbf"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.237794 14676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:18.255359 14676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:18.258033 14676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:18.258188 14676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:18.258684 14676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:18.259402 14676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:18.259445 14676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.259493 14676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:18.259521 14676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.267433 14676 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:38871
I20260927 18:38:18.267457 14804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:38871 every 8 connection(s)
I20260927 18:38:18.268252 14676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:18.273370 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14676
I20260927 18:38:18.273453 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:18.273700 6160 external_mini_cluster.cc:1428] Setting key dd8abf82ded95e391151cedb540f8ab0
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:18.280323 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.269779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:33267 (local address 127.6.4.62:42869)
0927 18:38:18.269889 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.269892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.270848 (+ 956us) server_negotiation.cc:407] Connection header received
0927 18:38:18.271688 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.271691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.271756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.271823 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.272948 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.273478 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.274696 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.274914 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.274997 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.277146 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:18.277148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:18.277150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:18.277172 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:18.278267 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:18.278684 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:18.278687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:18.278689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:18.278797 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:18.279092 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:18.279095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:18.279097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:18.279378 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:18.279468 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:18.280011 (+ 543us) server_negotiation.cc:299] Negotiation successful
0927 18:38:18.280162 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:38:18.281100 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.270098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:33267)
0927 18:38:18.270701 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.270747 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.271493 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.271997 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.272004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.272370 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.272811 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.272824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.273590 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.273594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.274601 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.274610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.274992 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.275040 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.275198 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.275690 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.275705 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.277004 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:18.278362 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:18.278368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:18.278381 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:18.278599 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:18.278896 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:18.278899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:18.278901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:18.279015 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:18.279463 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:18.279469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:18.279738 (+ 269us) client_negotiation.cc:769] Sending connection context
0927 18:38:18.279921 (+ 183us) client_negotiation.cc:240] Negotiation successful
0927 18:38:18.280246 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":126,"threads_started":1}
I20260927 18:38:18.282356 14805 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42869
I20260927 18:38:18.282550 14805 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:18.282968 14805 heartbeater.cc:511] Master 127.6.4.62:42869 requested a full tablet report, sending...
I20260927 18:38:18.283887 14346 ts_manager.cc:194] Registered new tserver with Master: 879db6a1bf3243cfb1ae62d23fe3f0ee (127.6.4.3:38871)
I20260927 18:38:18.284468 14346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:33267
I20260927 18:38:18.286978 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab.
Entry for principal oryx/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab.
WARNING: no policy specified for oryx/127.6.4.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab.
Entry for principal oryx/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab.
WARNING: no policy specified for oryx/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab.
Entry for principal oryx/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab.
I20260927 18:38:18.347468 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14408
I20260927 18:38:18.351941 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14542
I20260927 18:38:18.356283 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14676
I20260927 18:38:18.360261 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14314
I20260927 18:38:18.364907 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42869
--webserver_interface=127.6.4.62
--webserver_port=33075
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:18.470917 14814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:18.471143 14814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:18.471202 14814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:18.475090 14814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:18.475138 14814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:18.475164 14814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:18.475193 14814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:18.475222 14814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:18.480248 14814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=33075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:18.481123 14814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:18.482007 14814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:38:18.489759 14814 server_base.cc:1034] running on GCE node
W20260927 18:38:18.489795 14820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.489717 14819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.489717 14822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.490417 14814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:18.491349 14814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:18.492494 14814 hybrid_clock.cc:648] HybridClock initialized: now 1790534298492481 us; error 28 us; skew 500 ppm
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:18.494952 14814 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:38:18.495715 14814 webserver.cc:460] Webserver started at http://127.6.4.62:33075/ using document root <none> and password file <none>
I20260927 18:38:18.496223 14814 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:18.496300 14814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:18.499748 14814 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:18.501739 14829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.502863 14814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 18:38:18.502965 14814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "635bf104307a4eb885778a2cd158ca20"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "47cadab63000a62e456b69a8e67f78b2"
server_key_iv: "dff2764233ef029dc8ba6bbf6f31a03c"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.503432 14814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:18.516983 14814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:18.519549 14814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:18.519701 14814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:18.526140 14814 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:42869
I20260927 18:38:18.526151 14881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:42869 every 8 connection(s)
I20260927 18:38:18.527661 14814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:18.529749 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14814
I20260927 18:38:18.530215 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:43833
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:42869
--webserver_port=37175
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:38:18.532698 14882 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:38:18.534654 14882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Bootstrap starting.
I20260927 18:38:18.550115 14882 log.cc:824] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:18.556370 14882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:38:18.556676 14882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Bootstrap complete.
I20260927 18:38:18.560501 14882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:18.560880 14882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635bf104307a4eb885778a2cd158ca20, State: Initialized, Role: FOLLOWER
I20260927 18:38:18.561360 14882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:18.561465 14882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:18.561523 14882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:18.561609 14882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:38:18.563337 14882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:18.563621 14882 leader_election.cc:302] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 635bf104307a4eb885778a2cd158ca20; no voters:
I20260927 18:38:18.563889 14882 leader_election.cc:288] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:38:18.563979 14886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:38:18.564178 14886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [term 2 LEADER]: Becoming Leader. State: Replica: 635bf104307a4eb885778a2cd158ca20, State: Running, Role: LEADER
I20260927 18:38:18.564450 14886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } }
I20260927 18:38:18.565026 14882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:18.565395 14886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635bf104307a4eb885778a2cd158ca20. Latest consensus state: current_term: 2 leader_uuid: "635bf104307a4eb885778a2cd158ca20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } } }
I20260927 18:38:18.565495 14886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:18.566123 14890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:18.567668 14887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "635bf104307a4eb885778a2cd158ca20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635bf104307a4eb885778a2cd158ca20" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42869 } } }
I20260927 18:38:18.567771 14887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:18.569092 14890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:18.569782 14890 catalog_manager.cc:1259] Loaded cluster ID: 5770177db609429fb66e26b974029525
I20260927 18:38:18.569834 14890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:18.572063 14890 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:18.572714 14890 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 635bf104307a4eb885778a2cd158ca20: Loaded TSK: 0
I20260927 18:38:18.573876 14890 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:38:18.637518 14884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:18.637744 14884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:18.637796 14884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:18.641701 14884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:18.641750 14884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:18.641821 14884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:18.646854 14884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.1.keytab
--principal=oryx/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:43833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=37175
--tserver_master_addrs=127.6.4.62:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:18.647749 14884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:18.648707 14884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:18.657403 14908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.657524 14911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.657403 14909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.657843 14884 server_base.cc:1034] running on GCE node
I20260927 18:38:18.658193 14884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:18.658777 14884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:18.659950 14884 hybrid_clock.cc:648] HybridClock initialized: now 1790534298659930 us; error 36 us; skew 500 ppm
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:18.662433 14884 init.cc:377] Logged in from keytab as oryx/127.6.4.1@KRBTEST.COM (short username oryx)
I20260927 18:38:18.663203 14884 webserver.cc:460] Webserver started at http://127.6.4.1:37175/ using document root <none> and password file <none>
I20260927 18:38:18.663693 14884 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:18.663760 14884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:18.667368 14884 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:18.669288 14918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.670399 14884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:18.670490 14884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "7eed816b942a4e07ab5983da32c6d774"
format_stamp: "Formatted at 2026-09-27 18:38:17 on dist-test-slave-bzzj"
server_key: "06ff5bb55e0b4af36fc35acce6e81c63"
server_key_iv: "a869f54c09acefd244ed30ce6f1ee5b2"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.670958 14884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:18.687248 14884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:18.690129 14884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:18.690284 14884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:18.690867 14884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:18.691629 14884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:18.691671 14884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.691713 14884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:18.691741 14884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.700465 14884 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:43833
I20260927 18:38:18.700515 15031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:43833 every 8 connection(s)
I20260927 18:38:18.701440 14884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:18.707382 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 14884
I20260927 18:38:18.707854 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:35547
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:42869
--webserver_port=42849
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:38:18.713943 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:18.703213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:41029 (local address 127.6.4.62:42869)
0927 18:38:18.703641 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.703659 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.703874 (+ 215us) server_negotiation.cc:407] Connection header received
0927 18:38:18.705209 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.705220 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.705506 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.705772 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.706726 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.707986 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.709101 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.709771 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.709947 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.713279 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:41029: BlockingRecv error: recv got EOF from 127.6.4.1:41029 (error 108)
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":91,"threads_started":1}
I20260927 18:38:18.714026 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.703325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:41029)
0927 18:38:18.703733 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.703770 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.704871 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.705828 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.705834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.706145 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.706586 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.706603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.708128 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.708131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.708991 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.708999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.709901 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.709946 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.710090 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.710621 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.710639 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.713109 (+ 2470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":124,"threads_started":1}
W20260927 18:38:18.714558 15032 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:42869 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.4.62:42869: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:18.719377 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.714975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:60677)
0927 18:38:18.715079 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.715093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.715164 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.715376 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.715379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.715515 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.715591 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.715596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.716296 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.716299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.716821 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.716828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.717200 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.717216 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.717303 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.717695 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.717705 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.719273 (+ 1568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:38:18.719580 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:18.715033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:60677 (local address 127.6.4.62:42869)
0927 18:38:18.715126 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.715129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.715141 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:18.715243 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.715246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.715286 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.715355 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.715687 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.716210 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.716919 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.717125 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.717204 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.719467 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:60677: BlockingRecv error: recv got EOF from 127.6.4.1:60677 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:18.724514 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.720079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:35757)
0927 18:38:18.720186 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.720199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.720266 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.720459 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.720461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.720568 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.720628 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.720632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.721554 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.721556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.722087 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.722094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.722598 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.722611 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.722667 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.723042 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.723053 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.724438 (+ 1385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 18:38:18.724666 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:18.720180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:35757 (local address 127.6.4.62:42869)
0927 18:38:18.720287 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.720292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.720305 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:18.720337 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.720340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.720377 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.720451 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.720711 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.721473 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.722175 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.722507 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.722595 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.724567 (+ 1972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:35757: BlockingRecv error: recv got EOF from 127.6.4.1:35757 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260927 18:38:18.724673 15032 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:38:18.813544 15036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:18.813773 15036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:18.813817 15036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:18.817699 15036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:18.817750 15036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:18.817816 15036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:18.822639 15036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.2.keytab
--principal=oryx/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:35547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=42849
--tserver_master_addrs=127.6.4.62:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:18.823467 15036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:18.824357 15036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:18.832744 15041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.832749 15042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:18.832798 15044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:18.833096 15036 server_base.cc:1034] running on GCE node
I20260927 18:38:18.833431 15036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:18.833990 15036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:18.835135 15036 hybrid_clock.cc:648] HybridClock initialized: now 1790534298835122 us; error 27 us; skew 500 ppm
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534298, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:18.837535 15036 init.cc:377] Logged in from keytab as oryx/127.6.4.2@KRBTEST.COM (short username oryx)
I20260927 18:38:18.838260 15036 webserver.cc:460] Webserver started at http://127.6.4.2:42849/ using document root <none> and password file <none>
I20260927 18:38:18.838734 15036 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:18.838817 15036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:18.842248 15036 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:18.844158 15051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.845227 15036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:18.845320 15036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "98dc832659984a1cb661b3b5f45be3fc"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "67b8523f0f939e52ebf748bcd2a80f4d"
server_key_iv: "9dafdeb2ca6fc0939fc62b6dd11461f1"
server_key_version: "encryptionkey@0"
I20260927 18:38:18.845772 15036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:18.865237 15036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:18.867720 15036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:18.867872 15036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:18.868423 15036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:18.869145 15036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:18.869190 15036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.869237 15036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:18.869258 15036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:18.877205 15036 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:35547
I20260927 18:38:18.877264 15164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:35547 every 8 connection(s)
I20260927 18:38:18.878067 15036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:18.883793 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15036
I20260927 18:38:18.884294 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:38871
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:42869
--webserver_port=33895
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:38:18.888397 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:18.879576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:35531 (local address 127.6.4.62:42869)
0927 18:38:18.879689 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.879693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.880449 (+ 756us) server_negotiation.cc:407] Connection header received
0927 18:38:18.881631 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.881635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.881692 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.881764 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.882790 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.883313 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.884816 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.885038 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.885130 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.888271 (+ 3141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:35531: BlockingRecv error: recv got EOF from 127.6.4.2:35531 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:38:18.888921 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.879909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:35531)
0927 18:38:18.880324 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.880361 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.881440 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.881933 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.881940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.882242 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.882656 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.882668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.883421 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.883424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.884684 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.884695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.885130 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.885177 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.885351 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.885954 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.885968 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.888155 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":123,"threads_started":1}
W20260927 18:38:18.889317 15165 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:42869 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.4.62:42869: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:18.894052 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.889691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:43825)
0927 18:38:18.889792 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.889805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.889876 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.890078 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.890081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.890206 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.890281 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.890285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.890956 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.890960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.891460 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.891467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.891842 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.891856 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.891936 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.892358 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.892368 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.893950 (+ 1582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 18:38:18.894151 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:18.889740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:43825 (local address 127.6.4.62:42869)
0927 18:38:18.889852 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.889856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.889868 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:18.889954 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.889957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.889995 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.890056 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.890387 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.890868 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.891556 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.891765 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.891853 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.894062 (+ 2209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:43825: BlockingRecv error: recv got EOF from 127.6.4.2:43825 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:18 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:18.898854 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:18.894680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:38181)
0927 18:38:18.894762 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:18.894774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:18.894840 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:18.895034 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:18.895037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:18.895154 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:18.895211 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.895215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.895848 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.895851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.896375 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:18.896382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.896752 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.896766 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:18.896836 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.897200 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:18.897209 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:18.898737 (+ 1528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260927 18:38:18.898929 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:18.894733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:38181 (local address 127.6.4.62:42869)
0927 18:38:18.894833 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:18.894837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:18.894849 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:18.894915 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:18.894918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:18.894954 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:18.895015 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:18.895304 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.895761 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.896469 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:18.896678 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:18.896756 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:18.898846 (+ 2090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:38181: BlockingRecv error: recv got EOF from 127.6.4.2:38181 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260927 18:38:18.899211 15165 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:38:18.986899 15168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:18.987116 15168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:18.987179 15168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:18.991039 15168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:18.991087 15168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:18.991168 15168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:18.996044 15168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/krb5kdc/oryx_127.6.4.3.keytab
--principal=oryx/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:38871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=33895
--tserver_master_addrs=127.6.4.62:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:18.996944 15168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:18.997843 15168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:19.006697 15176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:19.006702 15173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:19.006702 15174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:19.006750 15168 server_base.cc:1034] running on GCE node
I20260927 18:38:19.007279 15168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:19.007862 15168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:19.009047 15168 hybrid_clock.cc:648] HybridClock initialized: now 1790534299009017 us; error 37 us; skew 500 ppm
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534299, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:19.011572 15168 init.cc:377] Logged in from keytab as oryx/127.6.4.3@KRBTEST.COM (short username oryx)
I20260927 18:38:19.012387 15168 webserver.cc:460] Webserver started at http://127.6.4.3:33895/ using document root <none> and password file <none>
I20260927 18:38:19.012863 15168 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:19.012926 15168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:19.016453 15168 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:19.018272 15183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:19.019341 15168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:19.019431 15168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "879db6a1bf3243cfb1ae62d23fe3f0ee"
format_stamp: "Formatted at 2026-09-27 18:38:18 on dist-test-slave-bzzj"
server_key: "f7a095a8f4f374133b7be4f17e25a09a"
server_key_iv: "582cc3cc5ccb6f84945f6bbdca6cadbf"
server_key_version: "encryptionkey@0"
I20260927 18:38:19.019933 15168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:19.049551 15168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:19.052448 15168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:19.052605 15168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:19.053126 15168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:19.053889 15168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:19.053941 15168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:19.053990 15168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:19.054019 15168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:19.062031 15168 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:38871
I20260927 18:38:19.062052 15296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:38871 every 8 connection(s)
I20260927 18:38:19.062906 15168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:19.070047 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15168
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:38:19.073086 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:19.064518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:34693 (local address 127.6.4.62:42869)
0927 18:38:19.064636 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:19.064639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:19.065488 (+ 849us) server_negotiation.cc:407] Connection header received
0927 18:38:19.066692 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:19.066695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:19.066741 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:19.066802 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:19.067905 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.068459 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.069571 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.069783 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.069874 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.072998 (+ 3124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:34693: BlockingRecv error: recv got EOF from 127.6.4.3:34693 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:38:19.073768 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:19.064896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:34693)
0927 18:38:19.065362 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:19.065401 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:19.066500 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:19.066957 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:19.066963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:19.067288 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:19.067772 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.067785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.068556 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.068559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.069457 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.069466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.069871 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.069918 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.070063 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.070583 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:19.070602 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:19.072912 (+ 2310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
W20260927 18:38:19.074245 15297 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:42869 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.4.62:42869: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:19.079241 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:19.070827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.6.4.62:42869)
0927 18:38:19.071032 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:19.071036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:19.071048 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:19.071084 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:19.071086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:19.071124 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:19.071199 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:19.071909 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.072511 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.073689 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.073971 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.074056 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.075575 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:19.075580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:19.075592 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:19.075616 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:19.077345 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:19.077783 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:19.077789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:19.077795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:19.077927 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:19.078159 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:19.078162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:19.078164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:19.078496 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:19.078668 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:19.078916 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 18:38:19.079120 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
I20260927 18:38:19.079283 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:19.074617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:35769)
0927 18:38:19.074711 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:19.074724 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:19.074783 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:19.074992 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:19.074994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:19.075123 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:19.075215 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.075220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.075963 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.075966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.076772 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.076782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.077166 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.077180 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.077276 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.077653 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:19.077664 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:19.079205 (+ 1541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:38:19.079413 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:19.074704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:35769 (local address 127.6.4.62:42869)
0927 18:38:19.074801 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:19.074804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:19.074838 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:38:19.074868 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:19.074871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:19.074908 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:19.074981 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:19.075307 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.075869 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.076880 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.077071 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.077142 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.079331 (+ 2189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:35769: BlockingRecv error: recv got EOF from 127.6.4.3:35769 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:19.084719 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:19.079769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:51959)
0927 18:38:19.079893 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:19.079904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:19.079958 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:19.080763 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:19.080939 (+ 176us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:19.081087 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:19.081169 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.081174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.081878 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.081881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.082375 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.082382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.082760 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.082773 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.082842 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.083191 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:19.083202 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:19.084643 (+ 1441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260927 18:38:19.084865 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:19.079831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:51959 (local address 127.6.4.62:42869)
0927 18:38:19.079933 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:19.079936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:19.079948 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:19.080628 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:19.080631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:19.080671 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:19.080734 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:19.081269 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.081786 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.082472 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.082684 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.082763 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.084764 (+ 2001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:51959: BlockingRecv error: recv got EOF from 127.6.4.3:51959 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260927 18:38:19.084867 15297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:19.729744 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:19.725150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:44259)
0927 18:38:19.725385 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:19.725397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:19.725476 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:19.725683 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:19.725685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:19.725857 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:19.725947 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.725952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.726615 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.726617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.727181 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.727187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.727566 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.727579 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.727632 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.728042 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:19.728052 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:19.729630 (+ 1578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
W20260927 18:38:19.729876 15303 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:19.725241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44259 (local address 127.6.4.62:42869)
0927 18:38:19.725441 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:19.725444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:19.725456 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:19.725548 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:19.725551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:19.725590 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:19.725659 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:19.726027 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.726539 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.727276 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.727499 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.727572 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.729762 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:44259: BlockingRecv error: recv got EOF from 127.6.4.1:44259 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":65,"threads_started":1}
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:19 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:19.904646 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:19.899718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:49385)
0927 18:38:19.899957 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:19.899971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:19.900068 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:19.900306 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:19.900309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:19.900493 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:19.900579 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.900585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.901295 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.901298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.901866 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:19.901874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.902262 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.902277 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:19.902358 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.902783 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:19.902793 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:19.904523 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
W20260927 18:38:19.904758 15303 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:19.899804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:49385 (local address 127.6.4.62:42869)
0927 18:38:19.899934 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:19.899938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:19.900030 (+ 92us) server_negotiation.cc:407] Connection header received
0927 18:38:19.900165 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:19.900168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:19.900214 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:19.900275 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:19.900681 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.901199 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.901972 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:19.902183 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:19.902263 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:19.904649 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:49385: BlockingRecv error: recv got EOF from 127.6.4.2:49385 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 27 18:38:20 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:20 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:20.090353 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:20.085338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:39057)
0927 18:38:20.085586 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:20.085598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:20.085696 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:20.085932 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:20.085935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:20.086099 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:20.086192 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:20.086197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.086882 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.086885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:20.087451 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:20.087459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.087823 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.087838 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:20.087897 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:20.088369 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:20.088381 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:20.090199 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":65,"threads_started":1}
W20260927 18:38:20.090430 15303 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:20.085414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:39057 (local address 127.6.4.62:42869)
0927 18:38:20.085548 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:20.085551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:20.085658 (+ 107us) server_negotiation.cc:407] Connection header received
0927 18:38:20.085761 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:20.085763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:20.085805 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:20.085866 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:20.086294 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.086798 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.087533 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.087751 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.087838 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:20.090334 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:39057: BlockingRecv error: recv got EOF from 127.6.4.3:39057 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 18:38:20 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:20 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:20.735585 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:20.730438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:44261)
0927 18:38:20.730672 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:20.730684 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:20.730766 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:20.731043 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:20.731045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:20.731255 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:20.731378 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:20.731386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.732320 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.732324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:20.732871 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:20.732879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.733226 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.733240 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:20.733294 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:20.733784 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:20.733796 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:20.735473 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":64,"threads_started":1}
W20260927 18:38:20.735704 15307 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:20.730494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44261 (local address 127.6.4.62:42869)
0927 18:38:20.730731 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:20.730735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:20.730752 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:38:20.730852 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:20.730856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:20.730911 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:20.730994 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:20.731497 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.732230 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.732956 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.733166 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.733231 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:20.735594 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:44261: BlockingRecv error: recv got EOF from 127.6.4.1:44261 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
Sep 27 18:38:20 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:20 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:20.910249 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:20.905297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:32791)
0927 18:38:20.905528 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:20.905540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:20.905628 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:20.905841 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:20.905843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:20.906002 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:20.906096 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:20.906101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.906852 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.906855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:20.907462 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:20.907472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.907911 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.907925 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:20.907980 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:20.908450 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:20.908462 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:20.910133 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":64,"threads_started":1}
W20260927 18:38:20.910391 15307 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:20.905367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:32791 (local address 127.6.4.62:42869)
0927 18:38:20.905487 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:20.905490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:20.905592 (+ 102us) server_negotiation.cc:407] Connection header received
0927 18:38:20.905698 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:20.905700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:20.905740 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:20.905808 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:20.906175 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.906723 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.907567 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:20.907839 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:20.907916 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:20.910277 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:32791: BlockingRecv error: recv got EOF from 127.6.4.2:32791 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 18:38:21 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:21 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:21.095959 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:21.090987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:36991)
0927 18:38:21.091241 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:21.091254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:21.091360 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:21.091582 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:21.091585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:21.091756 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:21.091849 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:21.091855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.092587 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.092590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:21.093161 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:21.093170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.093572 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.093589 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:21.093667 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:21.094122 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:21.094133 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:21.095819 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
W20260927 18:38:21.096050 15307 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:21.091101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:36991 (local address 127.6.4.62:42869)
0927 18:38:21.091212 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:21.091215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:21.091306 (+ 91us) server_negotiation.cc:407] Connection header received
0927 18:38:21.091439 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:21.091442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:21.091484 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:21.091538 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:21.091942 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.092484 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.093270 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.093487 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.093561 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:21.095956 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:36991: BlockingRecv error: recv got EOF from 127.6.4.3:36991 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 27 18:38:21 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:21 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:21.740919 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:21.736238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:53457)
0927 18:38:21.736491 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:21.736503 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:21.736595 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:21.736797 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:21.736799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:21.736960 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:21.737050 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:21.737055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.737718 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.737720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:21.738281 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:21.738288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.738621 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.738635 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:21.738687 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:21.739119 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:21.739129 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:21.740768 (+ 1639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":81,"threads_started":1}
W20260927 18:38:21.741031 15311 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:21.736287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:53457 (local address 127.6.4.62:42869)
0927 18:38:21.736493 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:21.736496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:21.736549 (+ 53us) server_negotiation.cc:407] Connection header received
0927 18:38:21.736667 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:21.736669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:21.736708 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:21.736771 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:21.737129 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.737645 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.738358 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.738563 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.738624 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:21.740924 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:53457: BlockingRecv error: recv got EOF from 127.6.4.1:53457 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
Sep 27 18:38:21 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:21 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:21.915987 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:21.911039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:51307)
0927 18:38:21.911286 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:21.911299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:21.911403 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:21.911618 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:21.911621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:21.911835 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:21.911928 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:21.911934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.912704 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.912708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:21.913263 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:21.913271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.913684 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.913700 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:21.913789 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:21.914211 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:21.914222 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:21.915832 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
W20260927 18:38:21.916051 15311 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:21.911096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51307 (local address 127.6.4.62:42869)
0927 18:38:21.911235 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:21.911240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:21.911365 (+ 125us) server_negotiation.cc:407] Connection header received
0927 18:38:21.911478 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:21.911481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:21.911526 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:21.911588 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:21.912043 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.912606 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.913363 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:21.913594 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:21.913689 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:21.915958 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:51307: BlockingRecv error: recv got EOF from 127.6.4.2:51307 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 27 18:38:22 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:22 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:22.101483 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:22.096680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:52207)
0927 18:38:22.096961 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:22.096972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:22.097065 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:22.097276 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:22.097277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:22.097446 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:22.097539 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:22.097545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.098241 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.098243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:22.098780 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:22.098787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.099141 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.099156 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:22.099219 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:22.099688 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:22.099699 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:22.101365 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":82,"threads_started":1}
W20260927 18:38:22.101619 15311 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:22.096747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:52207 (local address 127.6.4.62:42869)
0927 18:38:22.096865 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:22.096868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:22.097028 (+ 160us) server_negotiation.cc:407] Connection header received
0927 18:38:22.097145 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:22.097147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:22.097191 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:22.097249 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:22.097635 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.098155 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.098865 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.099069 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.099138 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:22.101509 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:52207: BlockingRecv error: recv got EOF from 127.6.4.3:52207 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 27 18:38:22 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:22 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:22.750051 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:22.741631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:39653)
0927 18:38:22.741869 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:22.741884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:22.741984 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:22.742188 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:22.742190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:22.742361 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:22.742452 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:22.742457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.743236 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.743240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:22.743807 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:22.743815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.744182 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.744197 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:22.744254 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:22.748188 (+ 3934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:22.748204 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:22.749939 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":60,"threads_started":1}
W20260927 18:38:22.750195 15315 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:22.741776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:39653 (local address 127.6.4.62:42869)
0927 18:38:22.742021 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:22.742024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:22.742035 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:38:22.742066 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:22.742068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:22.742108 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:22.742176 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:22.742532 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.743037 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.743897 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.744124 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.744189 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:22.750068 (+ 5879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:39653: BlockingRecv error: recv got EOF from 127.6.4.1:39653 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
Sep 27 18:38:22 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:22 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:22.921435 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:22.916689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:50603)
0927 18:38:22.916916 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:22.916929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:22.917021 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:22.917281 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:22.917283 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:22.917448 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:22.917539 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:22.917544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.918232 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.918234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:22.918768 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:22.918775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.919142 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.919156 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:22.919213 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:22.919649 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:22.919660 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:22.921321 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
W20260927 18:38:22.921551 15315 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:22.916754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:50603 (local address 127.6.4.62:42869)
0927 18:38:22.916881 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:22.916884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:22.916987 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:38:22.917134 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:22.917136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:22.917183 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:22.917254 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:22.917632 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.918155 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.918869 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:22.919077 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:22.919147 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:22.921436 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:50603: BlockingRecv error: recv got EOF from 127.6.4.2:50603 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 27 18:38:23 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:23 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:23.107295 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:23.102134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:42115)
0927 18:38:23.102404 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:23.102417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:23.102507 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:23.102711 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:23.102713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:23.102910 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:23.103005 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:23.103011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.103739 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.103742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:23.104348 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:23.104357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.104801 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.104817 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:23.104879 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:23.105325 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:23.105337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:23.107136 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
W20260927 18:38:23.107393 15315 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:23.102192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:42115 (local address 127.6.4.62:42869)
0927 18:38:23.102349 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:23.102354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:23.102474 (+ 120us) server_negotiation.cc:407] Connection header received
0927 18:38:23.102583 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:23.102585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:23.102630 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:23.102686 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:23.103130 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.103647 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.104466 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.104695 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.104785 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:23.107283 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:42115: BlockingRecv error: recv got EOF from 127.6.4.3:42115 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 27 18:38:23 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:23 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:23.755492 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:23.750743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:37471)
0927 18:38:23.750995 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:23.751008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:23.751099 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:23.751314 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:23.751316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:23.751480 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:23.751572 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:23.751577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.752252 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.752254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:23.752798 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:23.752804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.753134 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.753149 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:23.753204 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:23.753639 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:23.753664 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:23.755359 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
W20260927 18:38:23.755586 15319 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:23.750843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:37471 (local address 127.6.4.62:42869)
0927 18:38:23.751060 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:23.751063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:23.751076 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:23.751169 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:23.751171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:23.751210 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:23.751278 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:23.751651 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.752184 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.752873 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.753075 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.753137 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:23.755475 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:37471: BlockingRecv error: recv got EOF from 127.6.4.1:37471 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
Sep 27 18:38:23 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:23 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:23.927495 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:23.922144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:44013)
0927 18:38:23.922378 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:23.922390 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:23.922498 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:23.922726 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:23.922729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:23.922919 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:23.923011 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:23.923018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.923739 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.923743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:23.924326 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:23.924335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.924735 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.924752 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:23.924841 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:23.925322 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:23.925344 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:23.927363 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":63,"threads_started":1}
W20260927 18:38:23.927574 15319 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:23.922265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:44013 (local address 127.6.4.62:42869)
0927 18:38:23.922398 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:23.922402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:23.922469 (+ 67us) server_negotiation.cc:407] Connection header received
0927 18:38:23.922580 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:23.922583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:23.922631 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:23.922694 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:23.923122 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.923644 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.924432 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:23.924654 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:23.924737 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:23.927484 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:44013: BlockingRecv error: recv got EOF from 127.6.4.2:44013 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 18:38:24 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:24 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:24.113042 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:24.108005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:38865)
0927 18:38:24.108248 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:24.108260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:24.108352 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:24.108554 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:24.108556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:24.108750 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:24.108841 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:24.108846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.109541 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.109544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:24.110133 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:24.110142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.110543 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.110560 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:24.110622 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:24.111092 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:24.111103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:24.112907 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":86,"threads_started":1}
W20260927 18:38:24.113126 15319 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:24.108117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:38865 (local address 127.6.4.62:42869)
0927 18:38:24.108241 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:24.108245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:24.108322 (+ 77us) server_negotiation.cc:407] Connection header received
0927 18:38:24.108421 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:24.108423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:24.108465 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:24.108531 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:24.108925 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.109430 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.110245 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.110460 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.110533 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:24.113033 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:38865: BlockingRecv error: recv got EOF from 127.6.4.3:38865 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 18:38:24 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:24 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:24.761157 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:24.756169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:52601)
0927 18:38:24.756413 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:24.756427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:24.756515 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:24.756724 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:24.756725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:24.756897 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:24.756989 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:24.756994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.757684 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.757686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:24.758233 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:24.758239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.758600 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.758614 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:24.758667 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:24.759123 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:24.759133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:24.761047 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":62,"threads_started":1}
W20260927 18:38:24.761274 15323 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:24.756230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:52601 (local address 127.6.4.62:42869)
0927 18:38:24.756463 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:24.756467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:24.756481 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:38:24.756587 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:24.756589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:24.756630 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:24.756701 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:24.757078 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.757606 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.758315 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.758534 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.758606 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:24.761167 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:52601: BlockingRecv error: recv got EOF from 127.6.4.1:52601 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
Sep 27 18:38:24 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:24 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:24.933377 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:24.928166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:40937)
0927 18:38:24.928438 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:24.928451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:24.928568 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:24.928816 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:24.928819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:24.929008 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:24.929098 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:24.929104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.929841 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.929845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:24.930394 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:24.930403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.930799 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.930815 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:24.930892 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:24.931373 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:24.931387 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:24.933221 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
W20260927 18:38:24.933434 15323 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:24.928240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:40937 (local address 127.6.4.62:42869)
0927 18:38:24.928376 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:24.928381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:24.928517 (+ 136us) server_negotiation.cc:407] Connection header received
0927 18:38:24.928655 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:24.928658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:24.928703 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:24.928774 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:24.929209 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.929741 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.930503 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:24.930719 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:24.930802 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:24.933340 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:40937: BlockingRecv error: recv got EOF from 127.6.4.2:40937 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 18:38:25 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:25 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:25.118526 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:25.113699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:58851)
0927 18:38:25.113950 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:25.113963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:25.114061 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:25.114266 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:25.114268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:25.114429 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:25.114519 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:25.114524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.115206 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.115209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:25.115772 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:25.115781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.116152 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.116169 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:25.116237 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:25.116663 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:25.116674 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:25.118385 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
W20260927 18:38:25.118607 15323 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:25.113848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:58851 (local address 127.6.4.62:42869)
0927 18:38:25.113968 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:25.113971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:25.114023 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:38:25.114138 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:25.114141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:25.114183 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:25.114237 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:25.114596 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.115126 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.115863 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.116062 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.116149 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:25.118510 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:58851: BlockingRecv error: recv got EOF from 127.6.4.3:58851 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 27 18:38:25 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:25 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:25.766824 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:25.761849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:49019)
0927 18:38:25.762141 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:25.762154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:25.762245 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:25.762441 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:25.762443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:25.762605 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:25.762695 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:25.762699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.763437 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.763439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:25.763995 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:25.764003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.764432 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.764450 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:25.764528 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:25.764998 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:25.765010 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:25.766679 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":72,"threads_started":1}
W20260927 18:38:25.766914 15327 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:25.761922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:49019 (local address 127.6.4.62:42869)
0927 18:38:25.762149 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:25.762152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:25.762205 (+ 53us) server_negotiation.cc:407] Connection header received
0927 18:38:25.762307 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:25.762309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:25.762349 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:25.762419 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:25.762811 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.763351 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.764130 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.764346 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.764432 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:25.766797 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:49019: BlockingRecv error: recv got EOF from 127.6.4.1:49019 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
Sep 27 18:38:25 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:25 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:25.939119 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:25.934113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:50643)
0927 18:38:25.934397 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:25.934410 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:25.934505 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:25.934696 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:25.934698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:25.934877 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:25.934968 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:25.934973 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.935652 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.935655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:25.936263 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:25.936272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.936675 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.936689 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:25.936744 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:25.937183 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:25.937193 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:25.938974 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":84,"threads_started":1}
W20260927 18:38:25.939193 15327 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:25.934194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:50643 (local address 127.6.4.62:42869)
0927 18:38:25.934336 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:25.934339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:25.934468 (+ 129us) server_negotiation.cc:407] Connection header received
0927 18:38:25.934565 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:25.934567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:25.934608 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:25.934674 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:25.935051 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.935577 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.936383 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:25.936601 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:25.936685 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:25.939097 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:50643: BlockingRecv error: recv got EOF from 127.6.4.2:50643 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 27 18:38:26 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:26 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:26.124235 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:26.119241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:49541)
0927 18:38:26.119550 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:26.119563 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:26.119658 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:26.119852 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:26.119854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:26.120015 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:26.120125 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:26.120130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.120807 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.120809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:26.121351 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:26.121359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.121787 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.121831 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:26.121900 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:26.122342 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:26.122353 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:26.124120 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
W20260927 18:38:26.124360 15327 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:26.119297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:49541 (local address 127.6.4.62:42869)
0927 18:38:26.119422 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:26.119426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:26.119626 (+ 200us) server_negotiation.cc:407] Connection header received
0927 18:38:26.119719 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:26.119721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:26.119765 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:26.119828 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:26.120216 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.120728 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.121446 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.121659 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.121743 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:26.124254 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:49541: BlockingRecv error: recv got EOF from 127.6.4.3:49541 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 27 18:38:26 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:26 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:26.772584 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:26.767506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:45807)
0927 18:38:26.767751 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:26.767765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:26.767862 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:26.768154 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:26.768157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:26.768348 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:26.768442 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:26.768448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.769233 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.769236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:26.769777 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:26.769784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.770219 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.770237 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:26.770316 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:26.770753 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:26.770763 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:26.772472 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
W20260927 18:38:26.772723 15331 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:26.767648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:45807 (local address 127.6.4.62:42869)
0927 18:38:26.767860 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:26.767864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:26.767880 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:38:26.767925 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:26.767928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:26.767977 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:26.768122 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:26.768550 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.769134 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.769875 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.770124 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.770202 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:26.772596 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:45807: BlockingRecv error: recv got EOF from 127.6.4.1:45807 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
Sep 27 18:38:26 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:26 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:26.944926 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:26.939935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:47493)
0927 18:38:26.940205 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:26.940218 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:26.940310 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:26.940503 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:26.940505 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:26.940672 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:26.940798 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:26.940803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.941627 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.941629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:26.942208 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:26.942216 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.942601 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.942616 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:26.942672 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:26.943169 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:26.943184 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:26.944814 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":67,"threads_started":1}
W20260927 18:38:26.945032 15331 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:26.940035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:47493 (local address 127.6.4.62:42869)
0927 18:38:26.940176 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:26.940179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:26.940269 (+ 90us) server_negotiation.cc:407] Connection header received
0927 18:38:26.940371 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:26.940373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:26.940413 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:26.940478 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:26.940895 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.941543 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.942314 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:26.942532 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:26.942607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:26.944937 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:47493: BlockingRecv error: recv got EOF from 127.6.4.2:47493 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 18:38:27 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:27 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:27.130203 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:27.124931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:58623)
0927 18:38:27.125206 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:27.125217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:27.125322 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:27.125549 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:27.125552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:27.125738 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:27.125856 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:27.125863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.126587 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.126591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:27.127255 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:27.127264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.127753 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.127772 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:27.127856 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:27.128326 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:27.128338 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:27.130058 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
W20260927 18:38:27.130306 15331 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:27.125022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:58623 (local address 127.6.4.62:42869)
0927 18:38:27.125147 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:27.125151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:27.125295 (+ 144us) server_negotiation.cc:407] Connection header received
0927 18:38:27.125404 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:27.125407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:27.125454 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:27.125519 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:27.125959 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.126476 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.127367 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.127625 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.127734 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:27.130208 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:58623: BlockingRecv error: recv got EOF from 127.6.4.3:58623 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:38:27 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:27 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:27.782027 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:27.773213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:38165)
0927 18:38:27.773457 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:27.773470 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:27.773553 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:27.773816 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:27.773818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:27.774003 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:27.774094 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:27.774099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.774874 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.774877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:27.775416 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:27.775423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.775805 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.775819 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:27.775886 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:27.780039 (+ 4153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:27.780051 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:27.781912 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":63,"threads_started":1}
W20260927 18:38:27.782176 15335 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:27.773292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:38165 (local address 127.6.4.62:42869)
0927 18:38:27.773510 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:27.773513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:27.773528 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:38:27.773645 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:27.773649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:27.773711 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:27.773795 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:27.774182 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.774761 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.775512 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.775737 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.775812 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:27.782038 (+ 6226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:38165: BlockingRecv error: recv got EOF from 127.6.4.1:38165 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
Sep 27 18:38:27 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:27 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:27.950804 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:27.945655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:55277)
0927 18:38:27.945910 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:27.945923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:27.946020 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:27.946221 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:27.946223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:27.946401 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:27.946490 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:27.946494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.947152 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.947154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:27.947692 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:27.947698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.948129 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.948148 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:27.948240 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:27.948722 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:27.948754 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:27.950652 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
W20260927 18:38:27.950881 15335 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:27.945731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:55277 (local address 127.6.4.62:42869)
0927 18:38:27.945870 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:27.945874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:27.945990 (+ 116us) server_negotiation.cc:407] Connection header received
0927 18:38:27.946089 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:27.946091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:27.946135 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:27.946199 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:27.946569 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.947079 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.947778 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:27.948010 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:27.948127 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:27.950784 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:55277: BlockingRecv error: recv got EOF from 127.6.4.2:55277 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 18:38:28 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:28 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:28.136219 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:28.130868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:56711)
0927 18:38:28.131183 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:28.131195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:28.131289 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:28.131503 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:28.131506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:28.131682 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:28.131807 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:28.131813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.132612 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.132615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:28.133189 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:28.133198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.133598 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.133614 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:28.133690 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:28.134148 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:28.134160 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:28.136075 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":89,"threads_started":1}
W20260927 18:38:28.136332 15335 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:28.130938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:56711 (local address 127.6.4.62:42869)
0927 18:38:28.131058 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:28.131063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:28.131257 (+ 194us) server_negotiation.cc:407] Connection header received
0927 18:38:28.131357 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:28.131359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:28.131403 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:28.131465 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:28.131933 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.132520 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.133297 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.133514 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.133588 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:28.136224 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:56711: BlockingRecv error: recv got EOF from 127.6.4.3:56711 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 18:38:28 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:28 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:28.787674 15338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:28.782703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:46767)
0927 18:38:28.782942 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:28.782957 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:28.783053 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:28.783320 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:28.783323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:28.783518 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:28.783615 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:28.783620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.784486 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.784489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:28.785037 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:28.785044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.785390 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.785404 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:28.785458 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:28.785903 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:28.785914 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:28.787532 (+ 1618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":68,"threads_started":1}
W20260927 18:38:28.787851 15339 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:28.782766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:46767 (local address 127.6.4.62:42869)
0927 18:38:28.783015 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:28.783019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:28.783032 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:38:28.783144 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:28.783148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:28.783198 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:28.783291 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:28.783749 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.784387 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.785118 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.785330 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.785395 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:28.787656 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:46767: BlockingRecv error: recv got EOF from 127.6.4.1:46767 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
Sep 27 18:38:28 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:28 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:28.956316 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:28.951442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:42139)
0927 18:38:28.951734 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:28.951744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:28.951841 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:28.952110 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:28.952113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:28.952293 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:28.952381 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:28.952387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.953093 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.953097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:28.953640 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:28.953649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.954058 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.954073 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:28.954151 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:28.954563 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:28.954574 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:28.956195 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":66,"threads_started":1}
W20260927 18:38:28.956441 15339 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:28.951503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:42139 (local address 127.6.4.62:42869)
0927 18:38:28.951649 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:28.951654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:28.951811 (+ 157us) server_negotiation.cc:407] Connection header received
0927 18:38:28.951943 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:28.951947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:28.951990 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:28.952053 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:28.952487 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.952995 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.953760 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:28.953976 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:28.954059 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:28.956314 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:42139: BlockingRecv error: recv got EOF from 127.6.4.2:42139 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 18:38:29 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:29 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:29.141901 15341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:29.136931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:54183)
0927 18:38:29.137168 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:29.137181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:29.137267 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:29.137517 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:29.137520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:29.137688 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:29.137782 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:29.137787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.138439 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.138441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:29.138972 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:29.138978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.139434 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.139451 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:29.139540 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:29.140152 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:29.140167 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:29.141792 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":65,"threads_started":1}
W20260927 18:38:29.142005 15339 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:29.137081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:54183 (local address 127.6.4.62:42869)
0927 18:38:29.137191 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:29.137194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:29.137226 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:38:29.137349 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:29.137351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:29.137392 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:29.137448 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:29.137864 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.138367 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.139074 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.139345 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.139433 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:29.141897 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:54183: BlockingRecv error: recv got EOF from 127.6.4.3:54183 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 27 18:38:29 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:29 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:29.793052 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:29.788325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:38387)
0927 18:38:29.788555 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:29.788567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:29.788647 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:29.788840 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:29.788842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:29.789001 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:29.789091 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:29.789096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.789782 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.789784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:29.790326 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:29.790333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.790734 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.790749 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:29.790812 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:29.791297 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:29.791308 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:29.792943 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
W20260927 18:38:29.793183 15343 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:29.788379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:38387 (local address 127.6.4.62:42869)
0927 18:38:29.788597 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:29.788600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:29.788612 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:29.788717 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:29.788719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:29.788759 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:29.788828 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:29.789167 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.789691 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.790406 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.790646 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.790712 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:29.793058 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:38387: BlockingRecv error: recv got EOF from 127.6.4.1:38387 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
Sep 27 18:38:29 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:29 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:29.961687 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:29.956925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:38355)
0927 18:38:29.957189 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:29.957202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:29.957308 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:29.957503 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:29.957505 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:29.957671 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:29.957796 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:29.957801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.958498 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.958501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:29.959057 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:29.959065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.959429 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.959443 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:29.959498 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:29.959920 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:29.959930 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:29.961578 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
W20260927 18:38:29.961792 15343 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:29.957002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:38355 (local address 127.6.4.62:42869)
0927 18:38:29.957131 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:29.957137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:29.957263 (+ 126us) server_negotiation.cc:407] Connection header received
0927 18:38:29.957373 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:29.957375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:29.957418 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:29.957478 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:29.957875 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.958389 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.959148 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:29.959361 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:29.959433 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:29.961698 (+ 2265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:38355: BlockingRecv error: recv got EOF from 127.6.4.2:38355 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 27 18:38:30 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:30 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:30.147661 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:30.142591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:33663)
0927 18:38:30.142859 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:30.142873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:30.142979 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:30.143233 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:30.143236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:30.143471 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:30.143585 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:30.143594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.144316 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.144318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:30.144875 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:30.144883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.145250 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.145267 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:30.145336 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:30.145863 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:30.145876 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:30.147550 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
W20260927 18:38:30.147854 15343 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:30.142672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:33663 (local address 127.6.4.62:42869)
0927 18:38:30.142794 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:30.142799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:30.142945 (+ 146us) server_negotiation.cc:407] Connection header received
0927 18:38:30.143078 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:30.143080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:30.143128 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:30.143198 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:30.143689 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.144240 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.144954 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.145155 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.145219 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:30.147694 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:33663: BlockingRecv error: recv got EOF from 127.6.4.3:33663 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 27 18:38:30 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:30 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:30.798393 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:30.793660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:55243)
0927 18:38:30.793941 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:30.793954 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:30.794047 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:30.794241 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:30.794243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:30.794420 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:30.794510 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:30.794515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.795186 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.795188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:30.795731 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:30.795737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.796119 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.796136 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:30.796194 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:30.796631 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:30.796641 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:30.798274 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
W20260927 18:38:30.798516 15347 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:30.793730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:55243 (local address 127.6.4.62:42869)
0927 18:38:30.793958 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:30.793961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:30.794003 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:38:30.794107 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:30.794109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:30.794149 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:30.794218 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:30.794590 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.795111 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.795817 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.796031 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.796123 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:30.798404 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:55243: BlockingRecv error: recv got EOF from 127.6.4.1:55243 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":65,"threads_started":1}
Sep 27 18:38:30 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:30 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:30.967144 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:30.962350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:58433)
0927 18:38:30.962594 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:30.962606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:30.962698 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:30.962900 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:30.962902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:30.963068 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:30.963156 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:30.963161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.963842 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.963844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:30.964429 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:30.964436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.964875 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.964890 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:30.964951 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:30.965377 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:30.965388 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:30.967030 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
W20260927 18:38:30.967243 15347 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:30.962406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:58433 (local address 127.6.4.62:42869)
0927 18:38:30.962531 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:30.962535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:30.962665 (+ 130us) server_negotiation.cc:407] Connection header received
0927 18:38:30.962768 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:30.962770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:30.962812 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:30.962872 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:30.963232 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.963738 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.964521 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:30.964786 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:30.964862 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:30.967149 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:58433: BlockingRecv error: recv got EOF from 127.6.4.2:58433 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 18:38:31 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:31 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:31.153646 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:31.148349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:49021)
0927 18:38:31.148601 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:31.148615 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:31.148738 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:31.149002 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:31.149005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:31.149194 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:31.149286 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:31.149292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.150027 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.150030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:31.150582 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:31.150590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.150983 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.150999 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:31.151079 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:31.151572 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:31.151586 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:31.153504 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
W20260927 18:38:31.153748 15347 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:31.148419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:49021 (local address 127.6.4.62:42869)
0927 18:38:31.148560 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:31.148564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:31.148685 (+ 121us) server_negotiation.cc:407] Connection header received
0927 18:38:31.148857 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:31.148860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:31.148909 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:31.148968 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:31.149391 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.149925 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.150684 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.150903 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.150984 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:31.153634 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:49021: BlockingRecv error: recv got EOF from 127.6.4.3:49021 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 27 18:38:31 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:31 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:31.804523 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:31.799107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:41609)
0927 18:38:31.799385 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:31.799399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:31.799493 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:31.799752 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:31.799755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:31.799922 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:31.800017 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:31.800021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.800813 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.800816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:31.801560 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:31.801570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.802071 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.802088 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:31.802157 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:31.802610 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:31.802621 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:31.804354 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":100,"threads_started":1}
W20260927 18:38:31.804589 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:31.799188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:41609 (local address 127.6.4.62:42869)
0927 18:38:31.799409 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:31.799412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:31.799446 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:38:31.799562 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:31.799564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:31.799606 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:31.799677 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:31.800121 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.800693 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.801670 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.801950 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.802029 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:31.804480 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:41609: BlockingRecv error: recv got EOF from 127.6.4.1:41609 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":61,"threads_started":1}
Sep 27 18:38:31 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:31 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:31.972997 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:31.967803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:48821)
0927 18:38:31.968049 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:31.968064 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:31.968199 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:31.968478 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:31.968481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:31.968722 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:31.968838 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:31.968847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.969675 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.969679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:31.970258 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:31.970267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.970640 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.970655 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:31.970720 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:31.971137 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:31.971148 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:31.972864 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
W20260927 18:38:31.973073 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:31.967917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:48821 (local address 127.6.4.62:42869)
0927 18:38:31.968050 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:31.968055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:31.968158 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:38:31.968302 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:31.968305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:31.968356 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:31.968429 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:31.968956 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.969585 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.970355 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:31.970563 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:31.970635 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:31.972982 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:48821: BlockingRecv error: recv got EOF from 127.6.4.2:48821 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 18:38:32 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:32 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:32.159788 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:32.154536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:58489)
0927 18:38:32.154804 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:32.154817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:32.154917 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:32.155112 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:32.155113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:32.155278 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:32.155368 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:32.155373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.156196 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.156200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:32.156800 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:32.156811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.157225 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.157243 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:32.157324 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:32.157800 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:32.157814 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:32.159626 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
W20260927 18:38:32.159883 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:32.154595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:58489 (local address 127.6.4.62:42869)
0927 18:38:32.154724 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:32.154728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:32.154878 (+ 150us) server_negotiation.cc:407] Connection header received
0927 18:38:32.154977 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:32.154980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:32.155024 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:32.155084 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:32.155505 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.156069 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.156916 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.157127 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.157209 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:32.159768 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:58489: BlockingRecv error: recv got EOF from 127.6.4.3:58489 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:38:32 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:32 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:32.815476 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:32.805379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:37177)
0927 18:38:32.805630 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:32.805644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:32.805731 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:32.805938 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:32.805939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:32.806124 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:32.806236 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:32.806242 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.807051 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.807054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:32.807677 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:32.807685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.808136 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.808152 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:32.808225 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:32.813606 (+ 5381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:32.813618 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:32.815319 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
W20260927 18:38:32.815562 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:32.805464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:37177 (local address 127.6.4.62:42869)
0927 18:38:32.805727 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:32.805731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:32.805743 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:38:32.805797 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:32.805799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:32.805839 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:32.805912 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:32.806337 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.806951 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.807789 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.808031 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.808138 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:32.815448 (+ 7310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:37177: BlockingRecv error: recv got EOF from 127.6.4.1:37177 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":61,"threads_started":1}
Sep 27 18:38:32 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:32 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:32.978731 15356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:32.973669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:40563)
0927 18:38:32.973929 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:32.973943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:32.974054 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:32.974259 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:32.974261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:32.974454 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:32.974548 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:32.974554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.975259 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.975261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:32.975837 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:32.975846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.976230 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.976244 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:32.976351 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:32.976783 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:32.976795 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:32.978608 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
W20260927 18:38:32.978821 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:32.973739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:40563 (local address 127.6.4.62:42869)
0927 18:38:32.973861 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:32.973864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:32.973999 (+ 135us) server_negotiation.cc:407] Connection header received
0927 18:38:32.974118 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:32.974120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:32.974162 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:32.974235 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:32.974633 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.975178 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.975930 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:32.976165 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:32.976245 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:32.978729 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:40563: BlockingRecv error: recv got EOF from 127.6.4.2:40563 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 18:38:33 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:33 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.3@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:33.165457 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:33.160494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.3:42665)
0927 18:38:33.160786 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:33.160800 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:33.160902 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:33.161143 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:33.161145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:33.161310 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:33.161416 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:33.161422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.162102 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.162104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:33.162675 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:33.162683 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.163043 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.163058 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:33.163116 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:33.163595 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:33.163606 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:33.165335 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
W20260927 18:38:33.165561 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:33.160620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:42665 (local address 127.6.4.62:42869)
0927 18:38:33.160757 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:33.160760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:33.160863 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:38:33.160971 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:33.160973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:33.161017 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:33.161078 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:33.161500 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.162025 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.162759 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.162977 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.163048 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:33.165452 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.3:42665: BlockingRecv error: recv got EOF from 127.6.4.3:42665 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 27 18:38:33 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:33 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.1@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:33.821557 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:33.816393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.1:44515)
0927 18:38:33.816710 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:33.816725 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:33.816828 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:33.817161 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:33.817164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:33.817376 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:33.817496 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:33.817503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.818189 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.818191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:33.818739 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:33.818747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.819140 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.819156 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:33.819229 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:33.819751 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:33.819763 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:33.821432 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":75,"threads_started":1}
W20260927 18:38:33.821667 15359 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:33.816456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44515 (local address 127.6.4.62:42869)
0927 18:38:33.816679 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:33.816683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:33.816793 (+ 110us) server_negotiation.cc:407] Connection header received
0927 18:38:33.816981 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:33.816985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:33.817040 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:33.817128 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:33.817592 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.818109 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.818824 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.819035 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.819106 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:33.821548 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.1:44515: BlockingRecv error: recv got EOF from 127.6.4.1:44515 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":65,"threads_started":1}
Sep 27 18:38:33 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:38:33 dist-test-slave-bzzj krb5kdc[14301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.4.2@KRBTEST.COM for oryx/127.6.4.62@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:38:33.984632 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:33.979443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42869 (local address 127.6.4.2:49543)
0927 18:38:33.979769 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:33.979781 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:33.979884 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:33.980151 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:33.980154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:33.980348 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:33.980447 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:33.980453 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.981173 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.981176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:33.981737 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:33.981746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.982145 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.982161 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:33.982241 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:33.982664 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:33.982675 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:33.984491 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.4.62:42869: Server oryx/127.6.4.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
W20260927 18:38:33.984724 15359 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:33.979515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:49543 (local address 127.6.4.62:42869)
0927 18:38:33.979689 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:33.979693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:33.979851 (+ 158us) server_negotiation.cc:407] Connection header received
0927 18:38:33.979973 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:33.979976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:33.980026 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:33.980118 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:33.980557 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.981073 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.981846 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:33.982063 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:33.982145 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:33.984628 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.4.2:49543: BlockingRecv error: recv got EOF from 127.6.4.2:49543 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 18:38:34.072075 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14884
I20260927 18:38:34.076768 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15036
I20260927 18:38:34.081090 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15168
I20260927 18:38:34.085351 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 14814
2026-09-27T18:38:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17689 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T18:38:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:34Z Disabled control of system clock
I20260927 18:38:34.111968 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40565
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 18:38:34.216168 15364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:34.216406 15364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:34.216439 15364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:34.220310 15364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:34.220355 15364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:34.220369 15364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:34.220383 15364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:34.220396 15364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:34.224957 15364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:34.225961 15364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:34.227252 15364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:38:34.235517 15364 server_base.cc:1034] running on GCE node
W20260927 18:38:34.235571 15372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.235634 15369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.235530 15370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:34.236296 15364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:34.237329 15364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:34.238500 15364 hybrid_clock.cc:648] HybridClock initialized: now 1790534314238477 us; error 33 us; skew 500 ppm
I20260927 18:38:34.240309 15364 webserver.cc:460] Webserver started at http://127.6.4.62:36071/ using document root <none> and password file <none>
I20260927 18:38:34.240839 15364 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:34.240904 15364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:34.241097 15364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:34.243044 15364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "98c20edf134a4ae88768c58c61ddae23"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "2a7695b9b72daf246c0d69a6d4fb2cde"
server_key_iv: "03ba099072339fc649c1fe9d3217410a"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.243557 15364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "98c20edf134a4ae88768c58c61ddae23"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "2a7695b9b72daf246c0d69a6d4fb2cde"
server_key_iv: "03ba099072339fc649c1fe9d3217410a"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.246968 15364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:38:34.249279 15378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.250517 15364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:34.250633 15364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "98c20edf134a4ae88768c58c61ddae23"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "2a7695b9b72daf246c0d69a6d4fb2cde"
server_key_iv: "03ba099072339fc649c1fe9d3217410a"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.250733 15364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:34.281567 15364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:34.282265 15364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:34.282445 15364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:34.288679 15364 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40565
I20260927 18:38:34.288687 15430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40565 every 8 connection(s)
I20260927 18:38:34.289566 15364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:34.292318 15431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:34.296995 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15364
I20260927 18:38:34.297103 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:34.297394 6160 external_mini_cluster.cc:1428] Setting key 005cbf939d07850e4627438cfed106f4
I20260927 18:38:34.297817 15431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23: Bootstrap starting.
I20260927 18:38:34.300184 15431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:34.300884 15431 log.cc:824] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:34.302518 15431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23: No bootstrap required, opened a new log
I20260927 18:38:34.306017 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.298772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.6.4.62:40565)
0927 18:38:34.299404 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:34.299417 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:34.299451 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:38:34.300027 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:34.300043 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:34.300486 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:34.300777 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:34.301168 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.302110 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.303007 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.303471 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.303623 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.304453 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:34.304457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:34.304469 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:34.304830 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:34.304880 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:34.304899 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:34.304981 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:34.305109 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:38:34.305347 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":124,"threads_started":1}
I20260927 18:38:34.305912 15431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c20edf134a4ae88768c58c61ddae23" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40565 } }
I20260927 18:38:34.306102 15431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:34.306149 15431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c20edf134a4ae88768c58c61ddae23, State: Initialized, Role: FOLLOWER
I20260927 18:38:34.306555 15431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c20edf134a4ae88768c58c61ddae23" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40565 } }
I20260927 18:38:34.306679 15431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:34.306744 15431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:34.306836 15431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:34.307586 15431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c20edf134a4ae88768c58c61ddae23" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40565 } }
I20260927 18:38:34.307909 15431 leader_election.cc:302] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98c20edf134a4ae88768c58c61ddae23; no voters:
I20260927 18:38:34.308251 15431 leader_election.cc:288] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:34.308317 15436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:34.308507 15436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [term 1 LEADER]: Becoming Leader. State: Replica: 98c20edf134a4ae88768c58c61ddae23, State: Running, Role: LEADER
I20260927 18:38:34.308794 15436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c20edf134a4ae88768c58c61ddae23" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40565 } }
I20260927 18:38:34.309324 15431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:34.310369 15438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98c20edf134a4ae88768c58c61ddae23. Latest consensus state: current_term: 1 leader_uuid: "98c20edf134a4ae88768c58c61ddae23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c20edf134a4ae88768c58c61ddae23" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40565 } } }
I20260927 18:38:34.310484 15438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:34.310851 15445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:34.311092 15437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98c20edf134a4ae88768c58c61ddae23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c20edf134a4ae88768c58c61ddae23" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40565 } } }
I20260927 18:38:34.311196 15437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:34.313480 15445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:34.318564 15445 catalog_manager.cc:1347] Generated new cluster ID: 63b0e6f89c834acbbf9bf8663d8063f8
I20260927 18:38:34.318629 15445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:34.333616 15445 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:34.334777 15445 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:34.339872 15445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98c20edf134a4ae88768c58c61ddae23: Generated new TSK 0
I20260927 18:38:34.340553 15445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:38:34.343415 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40565
--builtin_ntp_servers=127.6.4.20:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 18:38:34.450615 15455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:34.450845 15455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:34.450881 15455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:34.454887 15455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:34.454937 15455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:34.454999 15455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:34.459623 15455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:34.460644 15455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:34.461941 15455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:34.470752 15460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.470705 15463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.470700 15461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:34.470875 15455 server_base.cc:1034] running on GCE node
I20260927 18:38:34.471305 15455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:34.471832 15455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:34.472981 15455 hybrid_clock.cc:648] HybridClock initialized: now 1790534314472962 us; error 30 us; skew 500 ppm
I20260927 18:38:34.474637 15455 webserver.cc:460] Webserver started at http://127.6.4.1:33945/ using document root <none> and password file <none>
I20260927 18:38:34.475155 15455 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:34.475212 15455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:34.475397 15455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:34.477276 15455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "44cf63e6cd1c463582eb26926cb6a49e"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "ba021566d7983e41250b479010cf4fec"
server_key_iv: "47d94fbc634ed46c55b6e7928146b5fa"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.477757 15455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "44cf63e6cd1c463582eb26926cb6a49e"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "ba021566d7983e41250b479010cf4fec"
server_key_iv: "47d94fbc634ed46c55b6e7928146b5fa"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.481163 15455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:38:34.483345 15469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.484477 15455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:34.484578 15455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "44cf63e6cd1c463582eb26926cb6a49e"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "ba021566d7983e41250b479010cf4fec"
server_key_iv: "47d94fbc634ed46c55b6e7928146b5fa"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.484661 15455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:34.492602 15455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:34.493130 15455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:34.493276 15455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:34.493765 15455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:34.494568 15455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:34.494612 15455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.494659 15455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:34.494688 15455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.502323 15455 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:35527
I20260927 18:38:34.502348 15582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:35527 every 8 connection(s)
I20260927 18:38:34.503093 15455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:34.509567 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15455
I20260927 18:38:34.509673 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:34.509953 6160 external_mini_cluster.cc:1428] Setting key 90283f4cfdb2146b0f216dba3ae565c6
I20260927 18:38:34.511659 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.504647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:51553 (local address 127.6.4.62:40565)
0927 18:38:34.504762 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:34.504765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:34.505717 (+ 952us) server_negotiation.cc:407] Connection header received
0927 18:38:34.507022 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:34.507025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:34.507080 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:34.507149 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:34.507964 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.508542 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.509784 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.510036 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.510131 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.511245 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:34.511249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:34.511251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:34.511264 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:34.511286 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:34.511296 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:34.511379 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:34.511507 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:38:34.511554 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:38:34.511914 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40565
--builtin_ntp_servers=127.6.4.20:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 18:38:34.512732 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.504984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40565 (local address 127.6.4.1:51553)
0927 18:38:34.505586 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:34.505623 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:34.506813 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:34.507312 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:34.507319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:34.507400 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:34.507825 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:34.507837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.508652 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.508654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:34.509659 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:34.509669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.510130 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.510179 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:34.510366 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.510991 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:34.510993 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:34.511029 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:34.511094 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:34.511374 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:34.511381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:34.511386 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:34.511505 (+ 119us) client_negotiation.cc:240] Negotiation successful
0927 18:38:34.511833 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
I20260927 18:38:34.513926 15583 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40565
I20260927 18:38:34.514132 15583 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:34.514586 15583 heartbeater.cc:511] Master 127.6.4.62:40565 requested a full tablet report, sending...
I20260927 18:38:34.516137 15395 ts_manager.cc:194] Registered new tserver with Master: 44cf63e6cd1c463582eb26926cb6a49e (127.6.4.1:35527)
I20260927 18:38:34.517657 15395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.1:51553
W20260927 18:38:34.616350 15586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:34.616576 15586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:34.616608 15586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:34.620487 15586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:34.620532 15586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:34.620589 15586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:34.625214 15586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:34.626151 15586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:34.627422 15586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:34.636384 15591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.636448 15594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.636394 15592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:34.636427 15586 server_base.cc:1034] running on GCE node
I20260927 18:38:34.636987 15586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:34.637537 15586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:34.638696 15586 hybrid_clock.cc:648] HybridClock initialized: now 1790534314638668 us; error 47 us; skew 500 ppm
I20260927 18:38:34.640372 15586 webserver.cc:460] Webserver started at http://127.6.4.2:43291/ using document root <none> and password file <none>
I20260927 18:38:34.640921 15586 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:34.640985 15586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:34.641170 15586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:34.643039 15586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "ddb5c0d9520449f6853be06ff93c1c73"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "4692ba448d8bd8abc2d690125cf3ec27"
server_key_iv: "98351c06992e6463559251d6da43476c"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.643532 15586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "ddb5c0d9520449f6853be06ff93c1c73"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "4692ba448d8bd8abc2d690125cf3ec27"
server_key_iv: "98351c06992e6463559251d6da43476c"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.646850 15586 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:38:34.649204 15600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.650300 15586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:34.650396 15586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "ddb5c0d9520449f6853be06ff93c1c73"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "4692ba448d8bd8abc2d690125cf3ec27"
server_key_iv: "98351c06992e6463559251d6da43476c"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.650484 15586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:34.665359 15586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:34.665915 15586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:34.666060 15586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:34.666556 15586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:34.667392 15586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:34.667438 15586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.667487 15586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:34.667515 15586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.675109 15586 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36377
I20260927 18:38:34.675137 15713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36377 every 8 connection(s)
I20260927 18:38:34.675874 15586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:34.677537 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15586
I20260927 18:38:34.677641 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:34.677923 6160 external_mini_cluster.cc:1428] Setting key 6cb8906ea7a1f281e8fcba3876d9c60d
I20260927 18:38:34.679870 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40565
--builtin_ntp_servers=127.6.4.20:37485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 18:38:34.684998 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.677463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:45821 (local address 127.6.4.62:40565)
0927 18:38:34.677606 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:34.677611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:34.678692 (+ 1081us) server_negotiation.cc:407] Connection header received
0927 18:38:34.680227 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:34.680231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:34.680273 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:34.680336 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:34.681378 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.682127 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.683176 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.683413 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.683488 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.684594 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:34.684597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:34.684599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:34.684610 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:34.684629 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:34.684636 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:34.684686 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:34.684831 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 18:38:34.684894 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:38:34.686043 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.677858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40565 (local address 127.6.4.2:45821)
0927 18:38:34.678548 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:34.678598 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:34.680011 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:34.680516 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:34.680524 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:34.680625 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:34.681219 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:34.681238 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.682243 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.682246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:34.683079 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:34.683088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.683510 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.683557 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:34.683732 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.684313 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:34.684315 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:34.684363 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:34.684438 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:34.684703 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:34.684711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:34.684716 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:34.684837 (+ 121us) client_negotiation.cc:240] Negotiation successful
0927 18:38:34.685120 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
I20260927 18:38:34.687005 15714 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40565
I20260927 18:38:34.687203 15714 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:34.687633 15714 heartbeater.cc:511] Master 127.6.4.62:40565 requested a full tablet report, sending...
I20260927 18:38:34.688588 15395 ts_manager.cc:194] Registered new tserver with Master: ddb5c0d9520449f6853be06ff93c1c73 (127.6.4.2:36377)
I20260927 18:38:34.689149 15395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.2:45821
W20260927 18:38:34.784459 15717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:34.784732 15717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:34.784782 15717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:34.788731 15717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:34.788775 15717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:34.788846 15717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:34.793320 15717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:34.794276 15717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:34.795531 15717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:34.804306 15723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.804312 15722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:34.804313 15725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:34.804646 15717 server_base.cc:1034] running on GCE node
I20260927 18:38:34.805008 15717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:34.805572 15717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:34.806715 15717 hybrid_clock.cc:648] HybridClock initialized: now 1790534314806689 us; error 37 us; skew 500 ppm
I20260927 18:38:34.808360 15717 webserver.cc:460] Webserver started at http://127.6.4.3:41335/ using document root <none> and password file <none>
I20260927 18:38:34.808844 15717 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:34.808904 15717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:34.809068 15717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:34.811019 15717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "a99de9349d9b471d935c82c4ffb2e992"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "ec34afafd85ce1f389c339c4d8d9b665"
server_key_iv: "5fa2e3a31150a0c65525f15a38cbc7c7"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.811530 15717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "a99de9349d9b471d935c82c4ffb2e992"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "ec34afafd85ce1f389c339c4d8d9b665"
server_key_iv: "5fa2e3a31150a0c65525f15a38cbc7c7"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.814814 15717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:38:34.817037 15731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.818135 15717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:34.818238 15717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "a99de9349d9b471d935c82c4ffb2e992"
format_stamp: "Formatted at 2026-09-27 18:38:34 on dist-test-slave-bzzj"
server_key: "ec34afafd85ce1f389c339c4d8d9b665"
server_key_iv: "5fa2e3a31150a0c65525f15a38cbc7c7"
server_key_version: "encryptionkey@0"
I20260927 18:38:34.818332 15717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:34.828586 15717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:34.829164 15717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:34.829319 15717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:34.829821 15717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:34.830654 15717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:34.830698 15717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.830749 15717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:34.830776 15717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:34.838550 15717 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:41233
I20260927 18:38:34.838570 15844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:41233 every 8 connection(s)
I20260927 18:38:34.839344 15717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:34.846534 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15717
I20260927 18:38:34.846645 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:34.846935 6160 external_mini_cluster.cc:1428] Setting key c61e8585f276cbd9a3e913eef2f39c4f
I20260927 18:38:34.847620 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.841019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:56109 (local address 127.6.4.62:40565)
0927 18:38:34.841178 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:34.841183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:34.842166 (+ 983us) server_negotiation.cc:407] Connection header received
0927 18:38:34.843429 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:34.843431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:34.843477 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:34.843548 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:34.844398 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.844919 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.845907 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.846109 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.846174 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.847156 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:34.847160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:34.847163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:34.847175 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:34.847198 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:34.847209 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:34.847268 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:34.847438 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:38:34.847494 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 18:38:34.848568 15847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:34.841425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40565 (local address 127.6.4.3:56109)
0927 18:38:34.842036 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:34.842074 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:34.843251 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:34.843686 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:34.843693 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:34.843777 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:34.844291 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:34.844305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.845003 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.845006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:34.845826 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:34.845834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:34.846169 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:34.846208 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:34.846333 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:34.846900 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:34.846902 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:34.846952 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:34.847012 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:34.847291 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:34.847298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:34.847304 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:34.847432 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 18:38:34.847691 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":102,"threads_started":1}
I20260927 18:38:34.849465 15845 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40565
I20260927 18:38:34.849670 15845 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:34.850128 15845 heartbeater.cc:511] Master 127.6.4.62:40565 requested a full tablet report, sending...
I20260927 18:38:34.851004 15395 ts_manager.cc:194] Registered new tserver with Master: a99de9349d9b471d935c82c4ffb2e992 (127.6.4.3:41233)
I20260927 18:38:34.851526 15395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.3:56109
I20260927 18:38:34.860368 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:38:34.862813 15434 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:34.862120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.6.4.62:40565)
0927 18:38:34.862223 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:34.862227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:34.862366 (+ 139us) server_negotiation.cc:407] Connection header received
0927 18:38:34.862460 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:34.862462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:34.862496 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:34.862555 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:34.862723 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58264: BlockingRecv error: recv got EOF from 127.0.0.1:58264 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:38:34.863179 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15455
I20260927 18:38:34.867488 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15586
I20260927 18:38:34.871877 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15717
I20260927 18:38:34.876896 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15364
2026-09-27T18:38:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (790 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T18:38:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:34Z Disabled control of system clock
I20260927 18:38:34.902423 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39757
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:39757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:38:35.004139 15856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:35.004366 15856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:35.004401 15856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:35.008244 15856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:35.008288 15856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:35.008301 15856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:35.008316 15856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:35.008335 15856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:35.012799 15856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:39757
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:35.013687 15856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:35.014971 15856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:35.022771 15864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.022809 15862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.022800 15861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:35.022866 15856 server_base.cc:1034] running on GCE node
I20260927 18:38:35.023530 15856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:35.024539 15856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:35.025687 15856 hybrid_clock.cc:648] HybridClock initialized: now 1790534315025672 us; error 28 us; skew 500 ppm
I20260927 18:38:35.027282 15856 webserver.cc:460] Webserver started at http://127.6.4.62:38213/ using document root <none> and password file <none>
I20260927 18:38:35.027791 15856 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:35.027858 15856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:35.028057 15856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:35.029947 15856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "ddb4d0e1a22847d4bdd7b8cee7142145"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "910298e6d6849bd55934d81e595433c0"
server_key_iv: "2d429b872799bc6f36ddad9b29242ad2"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.030450 15856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "ddb4d0e1a22847d4bdd7b8cee7142145"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "910298e6d6849bd55934d81e595433c0"
server_key_iv: "2d429b872799bc6f36ddad9b29242ad2"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.033762 15856 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:35.035848 15870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.036993 15856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:35.037093 15856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "ddb4d0e1a22847d4bdd7b8cee7142145"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "910298e6d6849bd55934d81e595433c0"
server_key_iv: "2d429b872799bc6f36ddad9b29242ad2"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.037176 15856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:35.041338 15856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:35.041863 15856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:35.042012 15856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:35.048110 15856 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:39757
I20260927 18:38:35.048117 15922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:39757 every 8 connection(s)
I20260927 18:38:35.048943 15856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:35.051759 15923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:35.056782 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145: Bootstrap starting.
I20260927 18:38:35.056993 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15856
I20260927 18:38:35.057101 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:35.057334 6160 external_mini_cluster.cc:1428] Setting key bb28b2ccfcaeb1ff731ef234737e19ea
I20260927 18:38:35.058990 15923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:35.059660 15923 log.cc:824] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:35.061581 15923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145: No bootstrap required, opened a new log
I20260927 18:38:35.062456 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.058487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.6.4.62:39757)
0927 18:38:35.059125 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:35.059145 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:35.059183 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:38:35.059760 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:35.059778 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:35.060244 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:35.060571 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:35.061274 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:35.061278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:35.061288 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:35.061298 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:35.061348 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:35.061365 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:35.061455 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:35.061573 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:38:35.061813 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":96,"threads_started":1}
I20260927 18:38:35.065210 15923 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39757 } }
I20260927 18:38:35.065411 15923 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:35.065455 15923 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb4d0e1a22847d4bdd7b8cee7142145, State: Initialized, Role: FOLLOWER
I20260927 18:38:35.065840 15923 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39757 } }
I20260927 18:38:35.065960 15923 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:35.066018 15923 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:35.066102 15923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:35.066860 15923 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39757 } }
I20260927 18:38:35.067154 15923 leader_election.cc:302] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddb4d0e1a22847d4bdd7b8cee7142145; no voters:
I20260927 18:38:35.067405 15923 leader_election.cc:288] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:35.067495 15928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:35.067682 15928 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [term 1 LEADER]: Becoming Leader. State: Replica: ddb4d0e1a22847d4bdd7b8cee7142145, State: Running, Role: LEADER
I20260927 18:38:35.067996 15928 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39757 } }
I20260927 18:38:35.068455 15923 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:35.069341 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39757 } } }
I20260927 18:38:35.069454 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:35.069729 15930 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddb4d0e1a22847d4bdd7b8cee7142145. Latest consensus state: current_term: 1 leader_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb4d0e1a22847d4bdd7b8cee7142145" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39757 } } }
I20260927 18:38:35.069794 15930 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:35.069799 15937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:35.072780 15937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:35.078420 15937 catalog_manager.cc:1347] Generated new cluster ID: ed78dd2551e44a29a348361060b112c2
I20260927 18:38:35.078474 15937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:35.089008 15937 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:35.089304 15937 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:35.094941 15937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddb4d0e1a22847d4bdd7b8cee7142145: Generated new TSK 0
I20260927 18:38:35.095742 15937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:38:35.106932 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39757
--builtin_ntp_servers=127.6.4.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:38:35.212275 15947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:35.212499 15947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:35.212535 15947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:35.216389 15947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:35.216428 15947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:35.216480 15947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:35.220918 15947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:35.221809 15947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:35.223021 15947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:35.231689 15952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.231689 15953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.231689 15955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:35.231997 15947 server_base.cc:1034] running on GCE node
I20260927 18:38:35.232411 15947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:35.233026 15947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:35.234193 15947 hybrid_clock.cc:648] HybridClock initialized: now 1790534315234178 us; error 24 us; skew 500 ppm
I20260927 18:38:35.240574 15947 webserver.cc:460] Webserver started at http://127.6.4.1:37593/ using document root <none> and password file <none>
I20260927 18:38:35.241159 15947 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:35.241223 15947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:35.241405 15947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:35.243427 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "4eb138cd331b4361a3f0fe5a99839008"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "ca9ea97c4daa944f536c433083565a2d"
server_key_iv: "bc2e7aed1b7c0bc35ccb5251c6087d96"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.243947 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "4eb138cd331b4361a3f0fe5a99839008"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "ca9ea97c4daa944f536c433083565a2d"
server_key_iv: "bc2e7aed1b7c0bc35ccb5251c6087d96"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.247349 15947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260927 18:38:35.249682 15961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.250800 15947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:35.250896 15947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "4eb138cd331b4361a3f0fe5a99839008"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "ca9ea97c4daa944f536c433083565a2d"
server_key_iv: "bc2e7aed1b7c0bc35ccb5251c6087d96"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.250982 15947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:35.255272 15947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:35.255795 15947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:35.255934 15947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:35.256444 15947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:35.257232 15947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:35.257278 15947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.257324 15947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:35.257354 15947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.265118 15947 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:41239
I20260927 18:38:35.265141 16074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:41239 every 8 connection(s)
I20260927 18:38:35.265888 15947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:35.271449 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.267474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:51555 (local address 127.6.4.62:39757)
0927 18:38:35.267590 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:35.267593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:35.268629 (+ 1036us) server_negotiation.cc:407] Connection header received
0927 18:38:35.269919 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:35.269922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:35.269969 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:35.270043 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:35.271030 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:35.271034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:35.271037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:35.271050 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:35.271069 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:35.271078 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:35.271152 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:35.271271 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 18:38:35.271316 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:38:35.272233 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.267817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39757 (local address 127.6.4.1:51555)
0927 18:38:35.268480 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:35.268516 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:35.269682 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:35.270197 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:35.270204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:35.270297 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:35.270785 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:35.270786 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:35.270822 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:35.270885 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:35.271143 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:35.271150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:35.271155 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:38:35.271302 (+ 147us) client_negotiation.cc:240] Negotiation successful
0927 18:38:35.271519 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260927 18:38:35.272457 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 15947
I20260927 18:38:35.272565 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:35.272859 6160 external_mini_cluster.cc:1428] Setting key e0b483566780be657946691aa97c7007
I20260927 18:38:35.273386 16075 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39757
I20260927 18:38:35.273581 16075 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:35.273944 16075 heartbeater.cc:511] Master 127.6.4.62:39757 requested a full tablet report, sending...
I20260927 18:38:35.274698 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39757
--builtin_ntp_servers=127.6.4.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 18:38:35.275444 15887 ts_manager.cc:194] Registered new tserver with Master: 4eb138cd331b4361a3f0fe5a99839008 (127.6.4.1:41239)
W20260927 18:38:35.378901 16078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:35.379109 16078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:35.379143 16078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:35.383245 16078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:35.383297 16078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:35.383348 16078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:35.388125 16078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:35.388964 16078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:35.390142 16078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:35.398693 16086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.398630 16083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:35.398648 16078 server_base.cc:1034] running on GCE node
W20260927 18:38:35.398758 16084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:35.399180 16078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:35.399700 16078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:35.400841 16078 hybrid_clock.cc:648] HybridClock initialized: now 1790534315400825 us; error 27 us; skew 500 ppm
I20260927 18:38:35.402411 16078 webserver.cc:460] Webserver started at http://127.6.4.2:37605/ using document root <none> and password file <none>
I20260927 18:38:35.402902 16078 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:35.402957 16078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:35.403132 16078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:35.404973 16078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "2ae37c42d34a4f0a9bfdd87afdb4658c"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "bc25ee2aa8942f0109fcae9ba9d0aeb1"
server_key_iv: "ff6baf060706fca88d73c1f37b039619"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.405488 16078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "2ae37c42d34a4f0a9bfdd87afdb4658c"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "bc25ee2aa8942f0109fcae9ba9d0aeb1"
server_key_iv: "ff6baf060706fca88d73c1f37b039619"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.408676 16078 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:38:35.410827 16092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.411892 16078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:35.411986 16078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "2ae37c42d34a4f0a9bfdd87afdb4658c"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "bc25ee2aa8942f0109fcae9ba9d0aeb1"
server_key_iv: "ff6baf060706fca88d73c1f37b039619"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.412070 16078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:35.416064 16078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:35.416580 16078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:35.416723 16078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:35.417178 16078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:35.417944 16078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:35.417986 16078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.418032 16078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:35.418061 16078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.425647 16078 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:37545
I20260927 18:38:35.425673 16205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:37545 every 8 connection(s)
I20260927 18:38:35.426427 16078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:35.430347 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 16078
I20260927 18:38:35.430470 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:35.430719 6160 external_mini_cluster.cc:1428] Setting key 960fc40082be052b23d684b183fa849b
I20260927 18:38:35.432440 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39757
--builtin_ntp_servers=127.6.4.20:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 18:38:35.432518 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.428141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:34395 (local address 127.6.4.62:39757)
0927 18:38:35.428250 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:35.428254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:35.429307 (+ 1053us) server_negotiation.cc:407] Connection header received
0927 18:38:35.430514 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:35.430518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:35.430565 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:35.430639 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:35.432074 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:35.432078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:35.432081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:35.432115 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:35.432138 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:35.432148 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:35.432222 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:35.432343 (+ 121us) server_negotiation.cc:299] Negotiation successful
0927 18:38:35.432384 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:38:35.433324 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.428564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39757 (local address 127.6.4.2:34395)
0927 18:38:35.429167 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:35.429203 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:35.430306 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:35.430805 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:35.430814 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:35.430906 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:35.431813 (+ 907us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:35.431815 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:35.431862 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:35.431927 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:35.432218 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:35.432225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:35.432231 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:38:35.432378 (+ 147us) client_negotiation.cc:240] Negotiation successful
0927 18:38:35.432602 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260927 18:38:35.434512 16206 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39757
I20260927 18:38:35.434723 16206 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:35.435178 16206 heartbeater.cc:511] Master 127.6.4.62:39757 requested a full tablet report, sending...
I20260927 18:38:35.436424 15887 ts_manager.cc:194] Registered new tserver with Master: 2ae37c42d34a4f0a9bfdd87afdb4658c (127.6.4.2:37545)
W20260927 18:38:35.547618 16209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:35.547845 16209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:35.547880 16209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:35.551854 16209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:35.551910 16209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:35.551963 16209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:35.556562 16209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:35.557454 16209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:35.558677 16209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:35.567667 16215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.567662 16214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:35.567749 16217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:35.567987 16209 server_base.cc:1034] running on GCE node
I20260927 18:38:35.568385 16209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:35.568986 16209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:35.570132 16209 hybrid_clock.cc:648] HybridClock initialized: now 1790534315570110 us; error 38 us; skew 500 ppm
I20260927 18:38:35.571732 16209 webserver.cc:460] Webserver started at http://127.6.4.3:43759/ using document root <none> and password file <none>
I20260927 18:38:35.572296 16209 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:35.572361 16209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:35.572546 16209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:35.574484 16209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "5df726d92c824221b7b8949e7f19f339"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "82bc46e70cfd056b867678246f6f9867"
server_key_iv: "831f448630289df6468ed44e255319f7"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.574990 16209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "5df726d92c824221b7b8949e7f19f339"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "82bc46e70cfd056b867678246f6f9867"
server_key_iv: "831f448630289df6468ed44e255319f7"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.578397 16209 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:35.580710 16223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.581910 16209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:35.582010 16209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "5df726d92c824221b7b8949e7f19f339"
format_stamp: "Formatted at 2026-09-27 18:38:35 on dist-test-slave-bzzj"
server_key: "82bc46e70cfd056b867678246f6f9867"
server_key_iv: "831f448630289df6468ed44e255319f7"
server_key_version: "encryptionkey@0"
I20260927 18:38:35.582103 16209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:35.586319 16209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:35.586865 16209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:35.587005 16209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:35.587472 16209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:35.588316 16209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:35.588363 16209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.588410 16209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:35.588438 16209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:35.596484 16209 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:45525
I20260927 18:38:35.596494 16336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:45525 every 8 connection(s)
I20260927 18:38:35.597298 16209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:35.599838 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 16209
I20260927 18:38:35.599946 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:35.600265 6160 external_mini_cluster.cc:1428] Setting key a8966ccd26d72f41ac5c520e4545b24d
I20260927 18:38:35.603386 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.598839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:42003 (local address 127.6.4.62:39757)
0927 18:38:35.598988 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:35.598992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:35.600068 (+ 1076us) server_negotiation.cc:407] Connection header received
0927 18:38:35.601986 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:35.601988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:35.602026 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:35.602098 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:35.603058 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:35.603060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:35.603062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:38:35.603072 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:35.603091 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:38:35.603098 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:35.603162 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:35.603251 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 18:38:35.603282 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 18:38:35.604171 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:35.599236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:39757 (local address 127.6.4.3:42003)
0927 18:38:35.599917 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:35.599970 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:35.601360 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:35.602284 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:35.602291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:35.602383 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:35.602834 (+ 451us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:38:35.602835 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:35.602873 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:38:35.602938 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:35.603158 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:35.603162 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:35.603166 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:38:35.603279 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:38:35.603466 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260927 18:38:35.605053 16337 heartbeater.cc:347] Connected to a master server at 127.6.4.62:39757
I20260927 18:38:35.605235 16337 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:35.605561 16337 heartbeater.cc:511] Master 127.6.4.62:39757 requested a full tablet report, sending...
I20260927 18:38:35.606381 15887 ts_manager.cc:194] Registered new tserver with Master: 5df726d92c824221b7b8949e7f19f339 (127.6.4.3:45525)
I20260927 18:38:35.613473 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:38:35.615994 15926 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:38:35.615154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.6.4.62:39757)
0927 18:38:35.615267 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:35.615272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:35.615468 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:38:35.615569 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:35.615572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:35.615603 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:35.615662 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:35.615884 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58280: BlockingRecv error: recv got EOF from 127.0.0.1:58280 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:38:35.616240 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15947
I20260927 18:38:35.620981 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 16078
I20260927 18:38:35.625213 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 16209
I20260927 18:38:35.629482 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 15856
2026-09-27T18:38:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (752 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:35 dist-test-slave-bzzj krb5kdc[16347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:35 dist-test-slave-bzzj krb5kdc[16347](info): set up 2 sockets
Sep 27 18:38:35 dist-test-slave-bzzj krb5kdc[16347](info): commencing operation
krb5kdc: starting...
W20260927 18:38:36.580655 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:36 dist-test-slave-bzzj krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:38:36.688916 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40583
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:36.788864 16360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:36.789093 16360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:36.789143 16360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:36.792959 16360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:38:36.793004 16360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:36.793030 16360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:36.793051 16360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:38:36.793079 16360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:38:36.797945 16360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:36.798784 16360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:36.799657 16360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:36.806846 16368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:36.806941 16366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:36.806854 16365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:36.807070 16360 server_base.cc:1034] running on GCE node
I20260927 18:38:36.807577 16360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:36.808542 16360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:36.809723 16360 hybrid_clock.cc:648] HybridClock initialized: now 1790534316809700 us; error 33 us; skew 500 ppm
Sep 27 18:38:36 dist-test-slave-bzzj krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534316, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:36.812184 16360 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:38:36.812894 16360 webserver.cc:460] Webserver started at http://127.6.4.62:42113/ using document root <none> and password file <none>
I20260927 18:38:36.813344 16360 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:36.813405 16360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:36.813549 16360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:36.815296 16360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "dc9208706c0343f09f9091643dceab24"
format_stamp: "Formatted at 2026-09-27 18:38:36 on dist-test-slave-bzzj"
server_key: "8836987c2b67fad97afa08f09d331861"
server_key_iv: "7b907220d79dc14228b78841530d1c8c"
server_key_version: "encryptionkey@0"
I20260927 18:38:36.815754 16360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "dc9208706c0343f09f9091643dceab24"
format_stamp: "Formatted at 2026-09-27 18:38:36 on dist-test-slave-bzzj"
server_key: "8836987c2b67fad97afa08f09d331861"
server_key_iv: "7b907220d79dc14228b78841530d1c8c"
server_key_version: "encryptionkey@0"
I20260927 18:38:36.818924 16360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:38:36.821009 16375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:36.822062 16360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:38:36.822156 16360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "dc9208706c0343f09f9091643dceab24"
format_stamp: "Formatted at 2026-09-27 18:38:36 on dist-test-slave-bzzj"
server_key: "8836987c2b67fad97afa08f09d331861"
server_key_iv: "7b907220d79dc14228b78841530d1c8c"
server_key_version: "encryptionkey@0"
I20260927 18:38:36.822243 16360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:36.836350 16360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:36.839038 16360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:36.839188 16360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:36.845265 16360 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40583
I20260927 18:38:36.845312 16427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40583 every 8 connection(s)
I20260927 18:38:36.846108 16360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:38:36.848640 16428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:36.853251 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 16360
I20260927 18:38:36.853376 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:38:36.853650 6160 external_mini_cluster.cc:1428] Setting key a21cb256014dd0f350d022dab719324b
I20260927 18:38:36.853899 16428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: Bootstrap starting.
I20260927 18:38:36.856185 16428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:36.856837 16428 log.cc:824] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:36.858423 16428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: No bootstrap required, opened a new log
Sep 27 18:38:36 dist-test-slave-bzzj krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:36.861857 16428 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9208706c0343f09f9091643dceab24" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40583 } }
I20260927 18:38:36.862054 16428 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:36.862109 16428 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9208706c0343f09f9091643dceab24, State: Initialized, Role: FOLLOWER
I20260927 18:38:36.862534 16428 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9208706c0343f09f9091643dceab24" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40583 } }
I20260927 18:38:36.862664 16428 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:38:36.862730 16428 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:38:36.862828 16428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:36.863758 16428 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9208706c0343f09f9091643dceab24" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40583 } }
I20260927 18:38:36.864127 16428 leader_election.cc:302] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9208706c0343f09f9091643dceab24; no voters:
I20260927 18:38:36.864413 16428 leader_election.cc:288] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:38:36.864739 16433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:36.864943 16433 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [term 1 LEADER]: Becoming Leader. State: Replica: dc9208706c0343f09f9091643dceab24, State: Running, Role: LEADER
I20260927 18:38:36.865293 16433 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9208706c0343f09f9091643dceab24" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40583 } }
I20260927 18:38:36.865517 16428 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:38:36.867028 16435 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9208706c0343f09f9091643dceab24. Latest consensus state: current_term: 1 leader_uuid: "dc9208706c0343f09f9091643dceab24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9208706c0343f09f9091643dceab24" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40583 } } }
I20260927 18:38:36.867060 16434 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc9208706c0343f09f9091643dceab24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9208706c0343f09f9091643dceab24" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40583 } } }
I20260927 18:38:36.867166 16435 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:36.867192 16434 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24 [sys.catalog]: This master's current role is: LEADER
I20260927 18:38:36.867019 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:36.854847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.6.4.62:40583)
0927 18:38:36.855244 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:36.855257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:36.855289 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:38:36.855861 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:36.855879 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:36.856224 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:36.856537 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:36.857269 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:36.858201 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:36.859034 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:36.859482 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:36.859618 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:36.861984 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:36.861988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:36.862000 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:36.862024 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:36.863884 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:36.864318 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:36.864324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:36.864330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:36.864481 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:36.864785 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:36.864787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:36.864789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:36.865256 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:36.865455 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:36.865734 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 18:38:36.866138 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":134,"threads_started":1}
W20260927 18:38:36.869995 16448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:38:36.870066 16448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:38:36.870117 16447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:38:36.870908 16447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:38:36.875586 16447 catalog_manager.cc:1347] Generated new cluster ID: 1e8e52ed9f014197a233eca9199867a6
I20260927 18:38:36.875645 16447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:38:36.887537 16447 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:38:36.888621 16447 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:38:36.908289 16447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: Generated new TSK 0
I20260927 18:38:36.908860 16447 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:38:36.937412 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40583
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:37.072535 16454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:37.072736 16454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:37.072773 16454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:37.076602 16454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:37.076643 16454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:37.076696 16454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:38:37.081511 16454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:37.082311 16454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:37.083137 16454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:37.091635 16460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:37.091651 16459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:37.091651 16462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:37.091938 16454 server_base.cc:1034] running on GCE node
I20260927 18:38:37.092314 16454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:37.092854 16454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:37.094010 16454 hybrid_clock.cc:648] HybridClock initialized: now 1790534317093992 us; error 32 us; skew 500 ppm
Sep 27 18:38:37 dist-test-slave-bzzj krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534317, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:37.096455 16454 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:38:37.097208 16454 webserver.cc:460] Webserver started at http://127.6.4.1:39423/ using document root <none> and password file <none>
I20260927 18:38:37.097675 16454 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:37.097733 16454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:37.097904 16454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:37.099673 16454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "267a1fd3cb8a426a95facc543e490783"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "84f6b2221fdc49e41d08fef7c204cce7"
server_key_iv: "f9bb1be025447ff79a8fb8dfe9c8ceec"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.100183 16454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "267a1fd3cb8a426a95facc543e490783"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "84f6b2221fdc49e41d08fef7c204cce7"
server_key_iv: "f9bb1be025447ff79a8fb8dfe9c8ceec"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.103328 16454 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:38:37.105444 16469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.106469 16454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:38:37.106572 16454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "267a1fd3cb8a426a95facc543e490783"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "84f6b2221fdc49e41d08fef7c204cce7"
server_key_iv: "f9bb1be025447ff79a8fb8dfe9c8ceec"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.106654 16454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:37.117650 16454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:37.120342 16454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:37.120491 16454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:37.120966 16454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:37.121676 16454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:37.121719 16454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.121765 16454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:37.121794 16454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.129499 16454 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:38437
I20260927 18:38:37.129536 16582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:38437 every 8 connection(s)
I20260927 18:38:37.130264 16454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:38:37.132740 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 16454
I20260927 18:38:37.132882 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:38:37.133189 6160 external_mini_cluster.cc:1428] Setting key aedc980835f663ce3722d4dde82ee6cd
Sep 27 18:38:37 dist-test-slave-bzzj krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534317, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:38:37.143680 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.131813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:45615 (local address 127.6.4.62:40583)
0927 18:38:37.131941 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.131944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.132757 (+ 813us) server_negotiation.cc:407] Connection header received
0927 18:38:37.134236 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.134238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.134278 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.134343 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:37.135428 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.136164 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.137684 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.137911 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.137988 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.140544 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:37.140547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:37.140549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:37.140571 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:37.141870 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.142397 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.142400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.142402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.142480 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.142795 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.142797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.142798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.142968 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:37.143036 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.143465 (+ 429us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.143563 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:38:37.144374 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.132164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40583 (local address 127.6.4.1:45615)
0927 18:38:37.132596 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.132646 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.134037 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.134507 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.134515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.134846 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:37.135266 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.135278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.136282 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.136285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.137560 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.137571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.137995 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.138042 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.138222 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.138754 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:37.138772 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:37.140388 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:37.141976 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:37.141982 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:37.141996 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:37.142290 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:37.142602 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:37.142604 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:37.142606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:37.142726 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:37.143037 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:37.143041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:37.143252 (+ 211us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.143414 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.143632 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:38:37.145495 16583 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40583
I20260927 18:38:37.145694 16583 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:37.146113 16583 heartbeater.cc:511] Master 127.6.4.62:40583 requested a full tablet report, sending...
I20260927 18:38:37.147512 16392 ts_manager.cc:194] Registered new tserver with Master: 267a1fd3cb8a426a95facc543e490783 (127.6.4.1:38437)
I20260927 18:38:37.148665 16392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:45615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:38:37.159013 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40583
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:37.261261 16588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:37.261482 16588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:37.261518 16588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:37.265446 16588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:37.265489 16588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:37.265542 16588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:38:37.270596 16588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:37.271436 16588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:37.272313 16588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:37.280934 16596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:37.280946 16594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:37.280946 16593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:37.281167 16588 server_base.cc:1034] running on GCE node
I20260927 18:38:37.281546 16588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:37.282094 16588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:37.283241 16588 hybrid_clock.cc:648] HybridClock initialized: now 1790534317283215 us; error 45 us; skew 500 ppm
Sep 27 18:38:37 dist-test-slave-bzzj krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534317, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:37.285682 16588 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:38:37.286410 16588 webserver.cc:460] Webserver started at http://127.6.4.2:46627/ using document root <none> and password file <none>
I20260927 18:38:37.286906 16588 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:37.286971 16588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:37.287168 16588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:37.289156 16588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "f2f4595bb9694f91a71c6f2c480ae9bd"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "055aaa547cb7733e0414783e1a7382ab"
server_key_iv: "5cacd8abee2c6973f048ee238d1a428d"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.289644 16588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "f2f4595bb9694f91a71c6f2c480ae9bd"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "055aaa547cb7733e0414783e1a7382ab"
server_key_iv: "5cacd8abee2c6973f048ee238d1a428d"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.292883 16588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:38:37.295058 16603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.296202 16588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:37.296310 16588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "f2f4595bb9694f91a71c6f2c480ae9bd"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "055aaa547cb7733e0414783e1a7382ab"
server_key_iv: "5cacd8abee2c6973f048ee238d1a428d"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.296401 16588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:37.310770 16588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:37.313238 16588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:37.313387 16588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:37.313894 16588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:37.314603 16588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:37.314647 16588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.314693 16588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:37.314723 16588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.322284 16588 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:45923
I20260927 18:38:37.322307 16716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:45923 every 8 connection(s)
I20260927 18:38:37.323024 16588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:38:37.323786 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 16588
I20260927 18:38:37.323894 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:38:37.324204 6160 external_mini_cluster.cc:1428] Setting key 2f70807e569d59142e3e52143059a881
Sep 27 18:38:37 dist-test-slave-bzzj krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534317, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:38:37.336292 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.324697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:59003 (local address 127.6.4.62:40583)
0927 18:38:37.324843 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.324847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.325669 (+ 822us) server_negotiation.cc:407] Connection header received
0927 18:38:37.327258 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.327262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.327316 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.327400 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:37.328427 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.329163 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.330357 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.330706 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.330822 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.333239 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:37.333243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:37.333244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:37.333271 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:37.334423 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.334848 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.334850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.334851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.334964 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.335209 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.335211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.335213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.335413 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:37.335480 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.335991 (+ 511us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.336134 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 18:38:37.336956 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.325076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40583 (local address 127.6.4.2:59003)
0927 18:38:37.325525 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.325575 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.327028 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.327523 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.327529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.327829 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:37.328301 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.328315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.329264 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.329267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.330230 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.330239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.330798 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.330833 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.330970 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.331481 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:37.331496 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:37.333091 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:37.334517 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:37.334522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:37.334532 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:37.334752 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:37.335028 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:37.335030 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:37.335032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:37.335146 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:37.335496 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:37.335502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:37.335766 (+ 264us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.335926 (+ 160us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.336237 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
I20260927 18:38:37.338084 16717 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40583
I20260927 18:38:37.338284 16717 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:37.338694 16717 heartbeater.cc:511] Master 127.6.4.62:40583 requested a full tablet report, sending...
I20260927 18:38:37.339651 16392 ts_manager.cc:194] Registered new tserver with Master: f2f4595bb9694f91a71c6f2c480ae9bd (127.6.4.2:45923)
I20260927 18:38:37.340220 16392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:59003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:38:37.350262 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40583
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:38:37.451247 16722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:38:37.451447 16722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:38:37.451483 16722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:38:37.455264 16722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:38:37.455308 16722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:38:37.455360 16722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:38:37.460170 16722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:38:37.460976 16722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:38:37.461815 16722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:38:37.470315 16730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:37.470327 16727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:38:37.470412 16728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:38:37.470534 16722 server_base.cc:1034] running on GCE node
I20260927 18:38:37.470963 16722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:38:37.471499 16722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:38:37.472637 16722 hybrid_clock.cc:648] HybridClock initialized: now 1790534317472623 us; error 29 us; skew 500 ppm
Sep 27 18:38:37 dist-test-slave-bzzj krb5kdc[16347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534317, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:38:37.475023 16722 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:38:37.475754 16722 webserver.cc:460] Webserver started at http://127.6.4.3:41011/ using document root <none> and password file <none>
I20260927 18:38:37.476267 16722 fs_manager.cc:359] Metadata directory not provided
I20260927 18:38:37.476328 16722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:38:37.476511 16722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:38:37.478443 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "2e9c478345a14cad8f072480933a71f1"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "8a243364b261ddce489cf95bd62f6c3b"
server_key_iv: "8a72bba67eac5912a42c27aeecea531d"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.478937 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "2e9c478345a14cad8f072480933a71f1"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "8a243364b261ddce489cf95bd62f6c3b"
server_key_iv: "8a72bba67eac5912a42c27aeecea531d"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.482326 16722 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 18:38:37.484694 16737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.485822 16722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:38:37.485918 16722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "2e9c478345a14cad8f072480933a71f1"
format_stamp: "Formatted at 2026-09-27 18:38:37 on dist-test-slave-bzzj"
server_key: "8a243364b261ddce489cf95bd62f6c3b"
server_key_iv: "8a72bba67eac5912a42c27aeecea531d"
server_key_version: "encryptionkey@0"
I20260927 18:38:37.486004 16722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:38:37.503741 16722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:38:37.506356 16722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:38:37.506516 16722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:38:37.507016 16722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:38:37.507752 16722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:38:37.507795 16722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.507843 16722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:38:37.507871 16722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:38:37.516315 16722 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37269
I20260927 18:38:37.516379 16850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37269 every 8 connection(s)
I20260927 18:38:37.517117 16722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:38:37.524838 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 16722
I20260927 18:38:37.524955 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:38:37.525251 6160 external_mini_cluster.cc:1428] Setting key a00e194e984bf7e462b6d371fc054611
Sep 27 18:38:37 dist-test-slave-bzzj krb5kdc[16347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534317, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:38:37.529392 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.518702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:47065 (local address 127.6.4.62:40583)
0927 18:38:37.518811 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.518815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.519554 (+ 739us) server_negotiation.cc:407] Connection header received
0927 18:38:37.520843 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.520846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.520895 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.520963 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:37.522012 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.522552 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.523604 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.523849 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.523939 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.526313 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:37.526316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:37.526318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:37.526342 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:37.527607 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.528044 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.528047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.528048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.528150 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.528444 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.528447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.528449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.528613 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:37.528692 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.529130 (+ 438us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.529239 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:38:37.530092 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.519012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40583 (local address 127.6.4.3:47065)
0927 18:38:37.519414 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.519453 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.520620 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.521100 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.521106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.521432 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:38:37.521859 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.521871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.522645 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.522647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.523488 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.523496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.523926 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.523964 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.524090 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.524583 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:38:37.524603 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:38:37.526145 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:38:37.527708 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:37.527714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:37.527728 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:37.527956 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:37.528226 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:38:37.528229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:38:37.528231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:38:37.528362 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:38:37.528701 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:38:37.528707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:38:37.528923 (+ 216us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.529079 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.529373 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":104,"threads_started":1}
I20260927 18:38:37.531038 16851 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40583
I20260927 18:38:37.531232 16851 heartbeater.cc:464] Registering TS with master...
I20260927 18:38:37.531646 16851 heartbeater.cc:511] Master 127.6.4.62:40583 requested a full tablet report, sending...
I20260927 18:38:37.532539 16392 ts_manager.cc:194] Registered new tserver with Master: 2e9c478345a14cad8f072480933a71f1 (127.6.4.3:37269)
I20260927 18:38:37.533046 16392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:47065
I20260927 18:38:37.538349 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:38:37.546200 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.540156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.6.4.62:40583)
0927 18:38:37.540267 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.540271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.540407 (+ 136us) server_negotiation.cc:407] Connection header received
0927 18:38:37.540494 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.540496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.540531 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.540587 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:38:37.541148 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.541622 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.542379 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.542591 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.542665 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.543741 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:38:37.543743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:38:37.543745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:38:37.543765 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:38:37.544964 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.545308 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.545312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.545314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.545380 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:38:37.545558 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:38:37.545559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:38:37.545561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:38:37.545734 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:38:37.545855 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.545999 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.546076 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:38:37.548596 16392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:38:37.551369 16392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:38:37.563454 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.558526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:38437 (local address 127.0.0.1:52240)
0927 18:38:37.558941 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.558988 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.559098 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.559592 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.559598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.559623 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:37.559737 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.559743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.560912 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.560915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.561737 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.561745 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.562878 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.562882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.563060 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.563098 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.563204 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.563325 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":39,"thread_start_us":67,"threads_started":1}
I20260927 18:38:37.563469 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.558685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.6.4.1:38437)
0927 18:38:37.559109 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.559120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.559143 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:38:37.559212 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.559217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.559424 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.559597 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:37.559871 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.560811 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.561848 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.562633 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.562727 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.562849 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.563239 (+ 390us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.563334 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":126,"threads_started":1}
I20260927 18:38:37.564469 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.559183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:45923 (local address 127.0.0.1:50618)
0927 18:38:37.559834 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.559845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.559920 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.560592 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.560595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.560611 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:37.560690 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.560696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.562109 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.562112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.563140 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.563148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.564056 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.564060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.564180 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.564201 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.564279 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.564353 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":104,"threads_started":1}
I20260927 18:38:37.566213 16517 tablet_service.cc:1467] Processing CreateTablet for tablet 4edde63d7af945ebaba2e3c86be2c537 (DEFAULT_TABLE table=test-table [id=dd13f0a08bf646919cace1814a860c80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:37.567159 16517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edde63d7af945ebaba2e3c86be2c537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:37.568542 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.558635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37269 (local address 127.0.0.1:36482)
0927 18:38:37.559305 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.559317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.559382 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.560467 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.560471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.560488 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:37.560578 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.560583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.561907 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.561910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.563041 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.563049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.568181 (+ 5132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.568184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.568261 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.568282 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.568355 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.568422 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":33,"thread_start_us":73,"threads_started":1}
I20260927 18:38:37.568596 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.559196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.6.4.3:37269)
0927 18:38:37.559926 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.559937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.559957 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:38:37.560159 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.560165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.560337 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.560469 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:37.560679 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.561740 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.563141 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.564197 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.564316 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.564416 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.568392 (+ 3976us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.568478 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":113,"threads_started":1}
I20260927 18:38:37.569052 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.559554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50618 (local address 127.6.4.2:45923)
0927 18:38:37.560193 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.560204 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.560227 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:38:37.560290 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.560293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.560456 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.560844 (+ 388us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:37.561002 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.562004 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.563255 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.563948 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.564038 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.564193 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.568829 (+ 4636us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.568925 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":78,"threads_started":1}
I20260927 18:38:37.571143 16785 tablet_service.cc:1467] Processing CreateTablet for tablet 4edde63d7af945ebaba2e3c86be2c537 (DEFAULT_TABLE table=test-table [id=dd13f0a08bf646919cace1814a860c80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:37.571326 16651 tablet_service.cc:1467] Processing CreateTablet for tablet 4edde63d7af945ebaba2e3c86be2c537 (DEFAULT_TABLE table=test-table [id=dd13f0a08bf646919cace1814a860c80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:38:37.572131 16651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edde63d7af945ebaba2e3c86be2c537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:37.572141 16785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edde63d7af945ebaba2e3c86be2c537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:38:37.573004 16868 tablet_bootstrap.cc:492] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: Bootstrap starting.
I20260927 18:38:37.575580 16868 tablet_bootstrap.cc:654] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:37.576318 16868 log.cc:824] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:37.577843 16868 tablet_bootstrap.cc:492] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: No bootstrap required, opened a new log
I20260927 18:38:37.578019 16868 ts_tablet_manager.cc:1397] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:38:37.578550 16870 tablet_bootstrap.cc:492] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Bootstrap starting.
I20260927 18:38:37.578945 16871 tablet_bootstrap.cc:492] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Bootstrap starting.
I20260927 18:38:37.581096 16871 tablet_bootstrap.cc:654] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:37.581655 16870 tablet_bootstrap.cc:654] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Neither blocks nor log segments found. Creating new log.
I20260927 18:38:37.581632 16868 raft_consensus.cc:348] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.581902 16868 raft_consensus.cc:374] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:37.581957 16868 raft_consensus.cc:729] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267a1fd3cb8a426a95facc543e490783, State: Initialized, Role: FOLLOWER
I20260927 18:38:37.582340 16871 log.cc:824] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:37.582389 16868 consensus_queue.cc:260] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.582732 16870 log.cc:824] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:38:37.583035 16583 heartbeater.cc:503] Master 127.6.4.62:40583 was elected leader, sending a full tablet report...
I20260927 18:38:37.583330 16868 ts_tablet_manager.cc:1428] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 18:38:37.584026 16871 tablet_bootstrap.cc:492] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: No bootstrap required, opened a new log
I20260927 18:38:37.584215 16871 ts_tablet_manager.cc:1397] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:38:37.584484 16870 tablet_bootstrap.cc:492] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: No bootstrap required, opened a new log
I20260927 18:38:37.584744 16870 ts_tablet_manager.cc:1397] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 18:38:37.587687 16870 raft_consensus.cc:348] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.587687 16871 raft_consensus.cc:348] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.587863 16871 raft_consensus.cc:374] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:37.587867 16870 raft_consensus.cc:374] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:38:37.587895 16871 raft_consensus.cc:729] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f4595bb9694f91a71c6f2c480ae9bd, State: Initialized, Role: FOLLOWER
I20260927 18:38:37.587929 16870 raft_consensus.cc:729] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e9c478345a14cad8f072480933a71f1, State: Initialized, Role: FOLLOWER
I20260927 18:38:37.588287 16871 consensus_queue.cc:260] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.588289 16870 consensus_queue.cc:260] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.588836 16851 heartbeater.cc:503] Master 127.6.4.62:40583 was elected leader, sending a full tablet report...
I20260927 18:38:37.589061 16717 heartbeater.cc:503] Master 127.6.4.62:40583 was elected leader, sending a full tablet report...
I20260927 18:38:37.589107 16871 ts_tablet_manager.cc:1428] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:38:37.589900 16870 ts_tablet_manager.cc:1428] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260927 18:38:37.631227 16584 tablet.cc:2369] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:37.767937 16852 tablet.cc:2369] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:38:37.824369 16718 tablet.cc:2369] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:38:37.839696 16876 raft_consensus.cc:482] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:38:37.839872 16876 raft_consensus.cc:504] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.841028 16876 leader_election.cc:288] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 267a1fd3cb8a426a95facc543e490783 (127.6.4.1:38437), f2f4595bb9694f91a71c6f2c480ae9bd (127.6.4.2:45923)
I20260927 18:38:37.844944 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.841275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:38437 (local address 127.6.4.3:60779)
0927 18:38:37.841395 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.841410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.841502 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.841840 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.841843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.841863 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:37.841984 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.841990 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.843032 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.843036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.843872 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.843881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.844560 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.844563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.844664 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.844692 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.844758 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.844815 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":19}
I20260927 18:38:37.844973 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.841332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:60779 (local address 127.6.4.1:38437)
0927 18:38:37.841451 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.841454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.841469 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:38:37.841610 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.841613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.841663 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.841742 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:37.842298 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.842933 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.843997 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.844462 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.844540 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.844590 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.844782 (+ 192us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.844848 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:38:37.845027 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.841637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:43725 (local address 127.6.4.2:45923)
0927 18:38:37.841770 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.841775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.841982 (+ 207us) server_negotiation.cc:407] Connection header received
0927 18:38:37.842094 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.842097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.842146 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.842207 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:38:37.842460 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.843096 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.844167 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.844556 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.844653 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.844709 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.844867 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.844927 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:38:37.845069 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.841540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:45923 (local address 127.6.4.3:43725)
0927 18:38:37.841904 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.841915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:38:37.841998 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:38:37.842232 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:38:37.842235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:38:37.842253 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:38:37.842361 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.842366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.843201 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.843204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.844036 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:38:37.844046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.844664 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.844666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:38:37.844755 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.844777 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:38:37.844847 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:38:37.844909 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":14,"thread_start_us":98,"threads_started":1}
I20260927 18:38:37.845618 16671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edde63d7af945ebaba2e3c86be2c537" candidate_uuid: "2e9c478345a14cad8f072480933a71f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" is_pre_election: true
I20260927 18:38:37.845912 16671 raft_consensus.cc:2454] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e9c478345a14cad8f072480933a71f1 in term 0.
I20260927 18:38:37.845979 16537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edde63d7af945ebaba2e3c86be2c537" candidate_uuid: "2e9c478345a14cad8f072480933a71f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "267a1fd3cb8a426a95facc543e490783" is_pre_election: true
I20260927 18:38:37.846225 16537 raft_consensus.cc:2454] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e9c478345a14cad8f072480933a71f1 in term 0.
I20260927 18:38:37.846338 16741 leader_election.cc:302] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e9c478345a14cad8f072480933a71f1, f2f4595bb9694f91a71c6f2c480ae9bd; no voters:
I20260927 18:38:37.846585 16876 raft_consensus.cc:2781] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:38:37.846663 16876 raft_consensus.cc:482] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:38:37.846737 16876 raft_consensus.cc:3037] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:37.847715 16876 raft_consensus.cc:504] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.848006 16876 leader_election.cc:288] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [CANDIDATE]: Term 1 election: Requested vote from peers 267a1fd3cb8a426a95facc543e490783 (127.6.4.1:38437), f2f4595bb9694f91a71c6f2c480ae9bd (127.6.4.2:45923)
I20260927 18:38:37.848414 16537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edde63d7af945ebaba2e3c86be2c537" candidate_uuid: "2e9c478345a14cad8f072480933a71f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "267a1fd3cb8a426a95facc543e490783"
I20260927 18:38:37.848518 16537 raft_consensus.cc:3037] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:37.849488 16537 raft_consensus.cc:2454] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e9c478345a14cad8f072480933a71f1 in term 1.
I20260927 18:38:37.849711 16671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edde63d7af945ebaba2e3c86be2c537" candidate_uuid: "2e9c478345a14cad8f072480933a71f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd"
I20260927 18:38:37.849741 16741 leader_election.cc:302] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267a1fd3cb8a426a95facc543e490783, 2e9c478345a14cad8f072480933a71f1; no voters:
I20260927 18:38:37.849828 16671 raft_consensus.cc:3037] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:38:37.849893 16876 raft_consensus.cc:2781] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:38:37.850101 16876 raft_consensus.cc:686] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 1 LEADER]: Becoming Leader. State: Replica: 2e9c478345a14cad8f072480933a71f1, State: Running, Role: LEADER
I20260927 18:38:37.850417 16876 consensus_queue.cc:237] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } }
I20260927 18:38:37.850992 16671 raft_consensus.cc:2454] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e9c478345a14cad8f072480933a71f1 in term 1.
I20260927 18:38:37.853259 16391 catalog_manager.cc:5495] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e9c478345a14cad8f072480933a71f1 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "2e9c478345a14cad8f072480933a71f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e9c478345a14cad8f072480933a71f1" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 } health_report { overall_health: UNKNOWN } } }
I20260927 18:38:37.920843 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.917470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36488 (local address 127.6.4.3:37269)
0927 18:38:37.917598 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:38:37.917602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:38:37.917617 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:38:37.917730 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:38:37.917733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:38:37.917787 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:38:37.917874 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:38:37.918115 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.918621 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.919443 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:38:37.919657 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:38:37.919740 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:38:37.920027 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:38:37.920462 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:38:37.920574 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 18:38:37.920643 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:38:37.920699 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:38:37.926159 16537 raft_consensus.cc:1261] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 1 FOLLOWER]: Refusing update from remote peer 2e9c478345a14cad8f072480933a71f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:37.926159 16671 raft_consensus.cc:1261] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 1 FOLLOWER]: Refusing update from remote peer 2e9c478345a14cad8f072480933a71f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:38:37.926846 16878 consensus_queue.cc:1035] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "267a1fd3cb8a426a95facc543e490783" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 38437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:37.927006 16876 consensus_queue.cc:1035] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f4595bb9694f91a71c6f2c480ae9bd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 45923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:38:37.940953 16390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:37.941078 16390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:38:37.942992 16390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc9208706c0343f09f9091643dceab24: Sending DeleteTablet for 3 replicas of tablet 4edde63d7af945ebaba2e3c86be2c537
I20260927 18:38:37.943692 16785 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edde63d7af945ebaba2e3c86be2c537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:37 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:36482
I20260927 18:38:37.943703 16517 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edde63d7af945ebaba2e3c86be2c537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:37 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:52240
I20260927 18:38:37.943790 16651 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edde63d7af945ebaba2e3c86be2c537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:38:37 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:50618
I20260927 18:38:37.944191 16891 tablet_replica.cc:330] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: stopping tablet replica
I20260927 18:38:37.944370 16892 tablet_replica.cc:330] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783: stopping tablet replica
I20260927 18:38:37.944473 16891 raft_consensus.cc:2229] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:38:37.944602 16892 raft_consensus.cc:2229] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:37.944671 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 16454
I20260927 18:38:37.944832 16892 raft_consensus.cc:2258] T 4edde63d7af945ebaba2e3c86be2c537 P 267a1fd3cb8a426a95facc543e490783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:37.945096 16893 tablet_replica.cc:330] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: stopping tablet replica
I20260927 18:38:37.945112 16891 raft_consensus.cc:2258] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:37.945335 16893 raft_consensus.cc:2229] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:38:37.945586 16893 raft_consensus.cc:2258] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:38:37.946616 16891 ts_tablet_manager.cc:1905] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:37.947199 16893 ts_tablet_manager.cc:1905] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:38:37.949131 16891 ts_tablet_manager.cc:1918] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:37.949220 16891 log.cc:1196] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/4edde63d7af945ebaba2e3c86be2c537
I20260927 18:38:37.949498 16891 ts_tablet_manager.cc:1939] T 4edde63d7af945ebaba2e3c86be2c537 P 2e9c478345a14cad8f072480933a71f1: Deleting consensus metadata
I20260927 18:38:37.949767 16893 ts_tablet_manager.cc:1918] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:38:37.949841 16893 log.cc:1196] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/4edde63d7af945ebaba2e3c86be2c537
I20260927 18:38:37.950091 16893 ts_tablet_manager.cc:1939] T 4edde63d7af945ebaba2e3c86be2c537 P f2f4595bb9694f91a71c6f2c480ae9bd: Deleting consensus metadata
I20260927 18:38:37.950520 16378 catalog_manager.cc:4841] TS 2e9c478345a14cad8f072480933a71f1 (127.6.4.3:37269): tablet 4edde63d7af945ebaba2e3c86be2c537 (table test-table [id=dd13f0a08bf646919cace1814a860c80]) successfully deleted
I20260927 18:38:37.951057 16379 catalog_manager.cc:4841] TS f2f4595bb9694f91a71c6f2c480ae9bd (127.6.4.2:45923): tablet 4edde63d7af945ebaba2e3c86be2c537 (table test-table [id=dd13f0a08bf646919cace1814a860c80]) successfully deleted
W20260927 18:38:37.952885 16379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:38437 (error 108)
I20260927 18:38:37.953079 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 16588
I20260927 18:38:37.957262 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:38:37.956985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:38437 (local address 127.0.0.1:52256)
0927 18:38:37.957134 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:38:37.957191 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:38437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260927 18:38:37.957417 16379 catalog_manager.cc:4584] TS 267a1fd3cb8a426a95facc543e490783 (127.6.4.1:38437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4edde63d7af945ebaba2e3c86be2c537: Network error: Client connection negotiation failed: client connection to 127.6.4.1:38437: connect: Connection refused (error 111)
I20260927 18:38:37.957930 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 16722
I20260927 18:38:37.962881 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 16360
2026-09-27T18:38:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2334 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:38:38 dist-test-slave-bzzj krb5kdc[16897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:38:38 dist-test-slave-bzzj krb5kdc[16897](info): set up 2 sockets
Sep 27 18:38:38 dist-test-slave-bzzj krb5kdc[16897](info): commencing operation
krb5kdc: starting...
W20260927 18:38:38.912618 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:38:38 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:38:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:38:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/rangeradmin_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/rangeradmin_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/rangerlookup_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/rangerlookup_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
I20260927 18:38:39.055024 6160 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFEZSEE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/postgres -l logfile start
2026-09-27 18:38:41.286 UTC [16927] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 18:38:41.286 UTC [16927] LOG: listening on IPv4 address "127.6.4.20", port 41089
2026-09-27 18:38:41.289 UTC [16927] LOG: listening on Unix socket "/tmp/.s.PGSQL.41089"
2026-09-27 18:38:41.314 UTC [16930] LOG: database system was shut down at 2026-09-27 18:38:40 UTC
2026-09-27 18:38:41.318 UTC [16927] LOG: database system is ready to accept connections
I20260927 18:38:42.180394 6160 mini_postgres.cc:96] Postgres bound to 41089
2026-09-27 18:38:42.186 UTC [16938] FATAL: database "slave" does not exist
127.6.4.20:41089 - accepting connections
I20260927 18:38:42.186781 6160 mini_ranger.cc:161] Starting Ranger...
I20260927 18:38:42.200026 6160 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 18:38:42.622792 6160 mini_ranger.cc:87] Created ranger Postgres database
I20260927 18:38:42.622882 6160 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-admin
2026-09-27 18:38:42,877 [I] DB FLAVOR :POSTGRES
2026-09-27 18:38:42,878 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:38:42,878 [I] Checking connection..
2026-09-27 18:38:42,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 18:38:43,124 [I] Checking connection passed.
2026-09-27 18:38:43,124 [I] --------- Verifying version history table ---------
2026-09-27 18:38:43,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:38:43,362 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:38:43,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:38:43,600 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:38:43,600 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 18:38:43,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 18:38:43.821 UTC [17002] WARNING: there is no transaction in progress
2026-09-27 18:38:43,826 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 18:38:43,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:38:44,052 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 18:38:44,052 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:38:44,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 18:38:44,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 18:38:44,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bzzj.c.gcp-upstream.internal','N') ;"
2026-09-27 18:38:44,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:38:44,962 [I] Table x_portal_user does not exist in database ranger
2026-09-27 18:38:44,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:38:45,208 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 18:38:45,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:38:45,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:38:45,677 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 18:38:45,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 18:38:46.548 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.562 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.577 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.591 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.605 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.621 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.670 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.676 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.688 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.697 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.706 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.719 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.727 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.732 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.738 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:46.744 UTC [17110] WARNING: there is no transaction in progress
2026-09-27 18:38:47,303 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 18:38:47,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 18:38:47,523 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 18:38:47,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:38:47,750 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 18:38:47,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:38:47,997 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 18:38:47,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:38:48,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:38:48,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bzzj.c.gcp-upstream.internal';"
2026-09-27 18:38:48,686 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 18:38:48,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 18:38:48,911 [I] DB_PATCHES have already been applied
I20260927 18:38:48.916755 6160 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFEZSEE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 18:38:48.916814 6160 mini_ranger.cc:193] Using host: 127.6.4.20
I20260927 18:38:48.919888 6160 mini_ranger.cc:227] Ranger admin URL: http://127.6.4.20:59599
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 6:38:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 18:38:49.912328 6160 mini_ranger.cc:160] Time spent starting Ranger: real 7.726s user 0.000s sys 0.008s
Sep 27, 2026 6:39:07 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 6:39:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 6:39:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 6:39:07 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 6:39:08 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 18:39:09.281934 6160 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/rangerkms_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/rangerkms_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.4.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.4.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 18:39:09 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 18:39:09.360340 6160 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 18:39:09.369235 6160 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 18:39:10.139951 6160 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 18:39:10.140029 6160 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms
I20260927 18:39:10.140149 6160 mini_ranger_kms.cc:227] Using postgres at 127.6.4.20:41089
2026-09-27 18:39:10,307 [I] DB FLAVOR :POSTGRES
2026-09-27 18:39:10,308 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:39:10,308 [I] Checking connection
2026-09-27 18:39:10,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 18:39:10,552 [I] connection success
2026-09-27 18:39:10,552 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 18:39:10,552 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 18:39:10,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 18:39:10,789 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 18:39:10,789 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:39:10,789 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 18:39:10,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:41089/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 18:39:11,031 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 18:39:11.480947 6160 mini_ranger_kms.cc:310] Created kms service
I20260927 18:39:11.688565 6160 mini_ranger_kms.cc:326] Created kudu user
I20260927 18:39:11.729101 6160 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 18:39:11 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534349, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:12.121539 6160 mini_ranger_kms.cc:384] Added ranger policy
I20260927 18:39:12.121685 6160 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 18:39:12.121707 6160 mini_ranger_kms.cc:240] Using host: 127.6.4.20
I20260927 18:39:12.125556 6160 mini_ranger_kms.cc:276] Ranger KMS PID: 17290
I20260927 18:39:12.125654 6160 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.4.20:60515
Sep 27, 2026 6:39:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:39:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:39:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 6:39:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 6:39:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 6:39:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 18:39:14.129889 6160 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.770s user 0.020s sys 0.131s
I20260927 18:39:14.129998 6160 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 18:39:14.130106 6160 mini_ranger_kms.cc:401] 127.6.4.20:60515/kms/v1/keys
Sep 27 18:39:14 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.4.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 18:39:14 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.4.20@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
Sep 27 18:39:15 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534354, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.4.20@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
Sep 27 18:39:17 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:39:17.450654 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:46803
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:46803
--ranger_config_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:60515/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:17.563439 17341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:17.563731 17341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:17.563771 17341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:17.568064 17341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:39:17.568168 17341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:17.568189 17341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:17.568205 17341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:39:17.568224 17341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:39:17.574096 17341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.4.20:60515/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:46803
--ranger_config_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:39:17.575385 17341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:17.576606 17341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:17.586201 17347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:17.586181 17346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:17.586184 17349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:17.586530 17341 server_base.cc:1034] running on GCE node
I20260927 18:39:17.587253 17341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:17.588820 17341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:17.590044 17341 hybrid_clock.cc:648] HybridClock initialized: now 1790534357590017 us; error 40 us; skew 500 ppm
Sep 27 18:39:17 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534357, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:17.593377 17341 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:39:17.594381 17341 webserver.cc:460] Webserver started at http://127.6.4.62:45939/ using document root <none> and password file <none>
I20260927 18:39:17.594930 17341 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:17.594986 17341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:17.595187 17341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:39:17 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534357, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:17.649914 17341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "0f9a2473d3584770912bc77f3b701ea2"
format_stamp: "Formatted at 2026-09-27 18:39:17 on dist-test-slave-bzzj"
server_key: "ba2e128f90f67b5f1102ace781177ac3"
server_key_iv: "b1cef3a15ef5720ed175cd5933914777"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:17.650530 17341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "0f9a2473d3584770912bc77f3b701ea2"
format_stamp: "Formatted at 2026-09-27 18:39:17 on dist-test-slave-bzzj"
server_key: "ba2e128f90f67b5f1102ace781177ac3"
server_key_iv: "b1cef3a15ef5720ed175cd5933914777"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:17.654511 17341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 18:39:17.672771 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:17.674158 17341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:39:17.674275 17341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "0f9a2473d3584770912bc77f3b701ea2"
format_stamp: "Formatted at 2026-09-27 18:39:17 on dist-test-slave-bzzj"
server_key: "ba2e128f90f67b5f1102ace781177ac3"
server_key_iv: "b1cef3a15ef5720ed175cd5933914777"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:17.674410 17341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:17.686210 17341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:17.689137 17341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:17.689321 17341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:17.697214 17341 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:46803
I20260927 18:39:17.697252 17408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:46803 every 8 connection(s)
I20260927 18:39:17.698195 17341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:39:17.701527 17409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:17.708132 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 17341
I20260927 18:39:17.708256 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:39:17.708374 17409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2: Bootstrap starting.
I20260927 18:39:17.708500 6160 external_mini_cluster.cc:1428] Setting key 900438a5badc51753b2886cdab3d50e9
I20260927 18:39:17.711752 17409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:17.712766 17409 log.cc:824] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:17.714907 17409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2: No bootstrap required, opened a new log
Sep 27 18:39:17 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:39:17.719048 17409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9a2473d3584770912bc77f3b701ea2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 46803 } }
I20260927 18:39:17.719349 17409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:17.719439 17409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9a2473d3584770912bc77f3b701ea2, State: Initialized, Role: FOLLOWER
I20260927 18:39:17.720065 17409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9a2473d3584770912bc77f3b701ea2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 46803 } }
I20260927 18:39:17.720273 17409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:39:17.720366 17409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:39:17.720487 17409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:17.721617 17409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9a2473d3584770912bc77f3b701ea2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 46803 } }
I20260927 18:39:17.721980 17409 leader_election.cc:302] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f9a2473d3584770912bc77f3b701ea2; no voters:
I20260927 18:39:17.722326 17409 leader_election.cc:288] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:39:17.722482 17414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:17.723537 17414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 0f9a2473d3584770912bc77f3b701ea2, State: Running, Role: LEADER
I20260927 18:39:17.723726 17409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:39:17.723953 17414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9a2473d3584770912bc77f3b701ea2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 46803 } }
I20260927 18:39:17.724416 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:17.709773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.6.4.62:46803)
0927 18:39:17.710249 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:17.710267 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:17.710311 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:39:17.711120 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:17.711144 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:17.711556 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:17.711931 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:17.712746 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:17.714029 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:17.715215 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:17.715886 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:17.716051 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:17.719123 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:17.719130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:17.719150 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:17.719185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:17.721135 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:17.721639 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:17.721646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:17.721652 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:17.721788 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:17.722074 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:17.722077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:17.722079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:17.722631 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:17.722828 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:17.723089 (+ 261us) server_negotiation.cc:299] Negotiation successful
0927 18:39:17.723366 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":138,"threads_started":1}
I20260927 18:39:17.725296 17414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f9a2473d3584770912bc77f3b701ea2. Latest consensus state: current_term: 1 leader_uuid: "0f9a2473d3584770912bc77f3b701ea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9a2473d3584770912bc77f3b701ea2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 46803 } } }
I20260927 18:39:17.725397 17414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:17.726140 17415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f9a2473d3584770912bc77f3b701ea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9a2473d3584770912bc77f3b701ea2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 46803 } } }
I20260927 18:39:17.726258 17415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:39:17.727254 17409 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:39:17.728871 17425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:39:17.734707 17425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:39:17.742280 17425 catalog_manager.cc:1347] Generated new cluster ID: da27ec6079e644fdb0cf10d29d626c2b
I20260927 18:39:17.742347 17425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:39:17.760802 17425 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:39:17.762192 17425 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:39:17.770418 17425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2: Generated new TSK 0
I20260927 18:39:17.771409 17425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:39:18.515568 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:60515/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:46803
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:18.654327 17454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:18.654605 17454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:18.654666 17454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:18.659788 17454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:18.659976 17454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:18.660077 17454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:39:18.668128 17454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.4.20:60515/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:39:18.669668 17454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:18.671022 17454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:18.683202 17461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.683735 17454 server_base.cc:1034] running on GCE node
W20260927 18:39:18.684207 17462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:18.684882 17464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:18.685334 17454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:18.685979 17454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:18.687155 17454 hybrid_clock.cc:648] HybridClock initialized: now 1790534358687117 us; error 55 us; skew 500 ppm
Sep 27 18:39:18 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:18.690737 17454 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:39:18.691857 17454 webserver.cc:460] Webserver started at http://127.6.4.1:46233/ using document root <none> and password file <none>
I20260927 18:39:18.692728 17454 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:18.692853 17454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:18.693130 17454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:39:18 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:18.714726 17454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "fc673964568c4e69b750c67b80866b61"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-bzzj"
server_key: "41fd108ee5d1a6d9e7d0f40c9789aa37"
server_key_iv: "e1916ab57f11ee51509d9b6e1f8b9050"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:18.715582 17454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "fc673964568c4e69b750c67b80866b61"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-bzzj"
server_key: "41fd108ee5d1a6d9e7d0f40c9789aa37"
server_key_iv: "e1916ab57f11ee51509d9b6e1f8b9050"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:18.719502 17454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260927 18:39:18.756526 17471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.758525 17454 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.000s
I20260927 18:39:18.758687 17454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "fc673964568c4e69b750c67b80866b61"
format_stamp: "Formatted at 2026-09-27 18:39:18 on dist-test-slave-bzzj"
server_key: "41fd108ee5d1a6d9e7d0f40c9789aa37"
server_key_iv: "e1916ab57f11ee51509d9b6e1f8b9050"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:18.758836 17454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:18.779227 17454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:18.782774 17454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:18.783028 17454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:18.783807 17454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:18.784933 17454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:18.784999 17454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:18.785058 17454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:18.785130 17454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 18:39:18.796059 17454 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:37437
I20260927 18:39:18.796142 17586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:37437 every 8 connection(s)
I20260927 18:39:18.797412 17454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:39:18.804558 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 17454
I20260927 18:39:18.804675 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:39:18.805022 6160 external_mini_cluster.cc:1428] Setting key 6bd73aa4cffb8cf3cdfade26bda3801d
Sep 27 18:39:18 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:39:18.817836 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.799566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:34167 (local address 127.6.4.62:46803)
0927 18:39:18.799888 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:18.799893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:18.803426 (+ 3533us) server_negotiation.cc:407] Connection header received
0927 18:39:18.804757 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:18.804761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:18.804823 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:18.804904 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:18.806681 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.807551 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.809301 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.809648 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.809743 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.812687 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:18.812691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:18.812693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:18.812723 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:18.814576 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.815187 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.815191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.815193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.815281 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:18.815660 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:18.815663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:18.815665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:18.815882 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:18.815965 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:18.817516 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0927 18:39:18.817657 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":74,"threads_started":1}
I20260927 18:39:18.818179 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:18.800048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:46803 (local address 127.6.4.1:34167)
0927 18:39:18.803226 (+ 3178us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:18.803284 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:18.804401 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:18.805218 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:18.805227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:18.805820 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:18.806516 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:18.806536 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.807699 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.807703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:18.809166 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:18.809177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:18.809878 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:18.809939 (+ 61us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:18.810155 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:18.810848 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:18.810868 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:18.812486 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:18.814706 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:18.814712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:18.814731 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:18.815074 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:18.815371 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:18.815374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:18.815376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:18.815561 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:18.816021 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:18.816028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:18.816394 (+ 366us) client_negotiation.cc:769] Sending connection context
0927 18:39:18.816650 (+ 256us) client_negotiation.cc:240] Negotiation successful
0927 18:39:18.817013 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2934,"thread_start_us":138,"threads_started":1}
I20260927 18:39:18.819700 17587 heartbeater.cc:347] Connected to a master server at 127.6.4.62:46803
I20260927 18:39:18.819971 17587 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:18.820631 17587 heartbeater.cc:511] Master 127.6.4.62:46803 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:39:18.822475 17373 ts_manager.cc:194] Registered new tserver with Master: fc673964568c4e69b750c67b80866b61 (127.6.4.1:37437)
I20260927 18:39:18.824074 17373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:34167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Sep 27 18:39:18 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:18.843159 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:60515/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:46803
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:39:18.980592 17593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:18.980860 17593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:18.980919 17593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:18.985265 17593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:18.985333 17593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:18.985414 17593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:39:18.991591 17593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.4.20:60515/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:39:18.993031 17593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:18.994501 17593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:19.007652 17602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:19.008571 17593 server_base.cc:1034] running on GCE node
W20260927 18:39:19.008790 17599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:19.008366 17600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:19.009438 17593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:19.010252 17593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:19.011449 17593 hybrid_clock.cc:648] HybridClock initialized: now 1790534359011374 us; error 93 us; skew 500 ppm
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.015743 17593 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:39:19.017081 17593 webserver.cc:460] Webserver started at http://127.6.4.2:45163/ using document root <none> and password file <none>
I20260927 18:39:19.017894 17593 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:19.018029 17593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:19.018301 17593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:19.039721 17593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "8fb4caadc76143ebbba6d83e6efc90dd"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-bzzj"
server_key: "a89190645022f960cce879d3e647812c"
server_key_iv: "5f1a8daf1ba61462c4e489bc4633b462"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:19.040557 17593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "8fb4caadc76143ebbba6d83e6efc90dd"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-bzzj"
server_key: "a89190645022f960cce879d3e647812c"
server_key_iv: "5f1a8daf1ba61462c4e489bc4633b462"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:19.045851 17593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 18:39:19.062517 17609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.064545 17593 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260927 18:39:19.064702 17593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "8fb4caadc76143ebbba6d83e6efc90dd"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-bzzj"
server_key: "a89190645022f960cce879d3e647812c"
server_key_iv: "5f1a8daf1ba61462c4e489bc4633b462"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:19.064853 17593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:19.095755 17593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:19.101300 17593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:19.101506 17593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:19.102196 17593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:19.103266 17593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:19.103323 17593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.103372 17593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:19.103399 17593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.113679 17593 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:33121
I20260927 18:39:19.113726 17722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:33121 every 8 connection(s)
I20260927 18:39:19.114799 17593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:39:19.122841 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 17593
I20260927 18:39:19.122975 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:39:19.123310 6160 external_mini_cluster.cc:1428] Setting key 82bbba4e7a08d34ae6c253f9cc6dab06
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:39:19.129974 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.116872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:51321 (local address 127.6.4.62:46803)
0927 18:39:19.117021 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.117025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.117976 (+ 951us) server_negotiation.cc:407] Connection header received
0927 18:39:19.119102 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.119105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.119154 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.119225 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.120629 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.121157 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.122885 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.123227 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.123329 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.126058 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.126062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.126065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.126094 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.127524 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.128200 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.128204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.128206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.128296 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.128684 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.128687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.128688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.128855 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.128923 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.129717 (+ 794us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.129829 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:39:19.137359 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.117284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:46803 (local address 127.6.4.2:51321)
0927 18:39:19.117786 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.117846 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.118871 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.119419 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.119427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.119778 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:19.120480 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.120498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.121293 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.121297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.122738 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.122750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.123464 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.123526 (+ 62us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.123727 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.124405 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:19.124425 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:19.125900 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:19.127651 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:19.127658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:19.127678 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:19.128049 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:19.128386 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:19.128390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:19.128392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:19.128598 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:19.128962 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:19.128971 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:19.129397 (+ 426us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.129628 (+ 231us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.129967 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260927 18:39:19.139068 17723 heartbeater.cc:347] Connected to a master server at 127.6.4.62:46803
I20260927 18:39:19.139323 17723 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:19.139928 17723 heartbeater.cc:511] Master 127.6.4.62:46803 requested a full tablet report, sending...
I20260927 18:39:19.141170 17373 ts_manager.cc:194] Registered new tserver with Master: 8fb4caadc76143ebbba6d83e6efc90dd (127.6.4.2:33121)
I20260927 18:39:19.141901 17373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:51321
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:39:19.163715 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:60515/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:46803
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
W20260927 18:39:19.327754 17728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:39:19.328061 17728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:39:19.328151 17728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:39:19.334049 17728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:39:19.334153 17728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:39:19.334239 17728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:39:19.342164 17728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.4.20:60515/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:39:19.343573 17728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:39:19.344903 17728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:39:19.358678 17734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:19.358680 17736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:39:19.359272 17733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:39:19.359494 17728 server_base.cc:1034] running on GCE node
I20260927 18:39:19.360222 17728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:39:19.361078 17728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:39:19.362265 17728 hybrid_clock.cc:648] HybridClock initialized: now 1790534359362195 us; error 87 us; skew 500 ppm
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:39:19.365756 17728 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:39:19.366830 17728 webserver.cc:460] Webserver started at http://127.6.4.3:38055/ using document root <none> and password file <none>
I20260927 18:39:19.367523 17728 fs_manager.cc:359] Metadata directory not provided
I20260927 18:39:19.367609 17728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:39:19.367880 17728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:19.401026 17728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "34afbf33ec3641798a38393322bdfb95"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-bzzj"
server_key: "ad9f7e036b1ac8abce3536bf18878096"
server_key_iv: "ecb242beb42c2075649ba7c71f7074c6"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:19.401774 17728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "34afbf33ec3641798a38393322bdfb95"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-bzzj"
server_key: "ad9f7e036b1ac8abce3536bf18878096"
server_key_iv: "ecb242beb42c2075649ba7c71f7074c6"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:19.409586 17728 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.000s
I20260927 18:39:19.437430 17743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.439045 17728 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 18:39:19.439168 17728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "34afbf33ec3641798a38393322bdfb95"
format_stamp: "Formatted at 2026-09-27 18:39:19 on dist-test-slave-bzzj"
server_key: "ad9f7e036b1ac8abce3536bf18878096"
server_key_iv: "ecb242beb42c2075649ba7c71f7074c6"
server_key_version: "kuduclusterkey@0"
I20260927 18:39:19.439291 17728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:39:19.463253 17728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:39:19.466681 17728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:39:19.466879 17728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:39:19.467517 17728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:39:19.468727 17728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:39:19.468786 17728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.468842 17728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:39:19.468869 17728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:39:19.482474 17728 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:41093
I20260927 18:39:19.483683 17728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:39:19.483986 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 17728
I20260927 18:39:19.484143 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:39:19.484483 6160 external_mini_cluster.cc:1428] Setting key 87b554294130e281e41f1c9532adaabc
I20260927 18:39:19.486985 17856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:41093 every 8 connection(s)
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534359, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:39:19.512854 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.488932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:39615 (local address 127.6.4.62:46803)
0927 18:39:19.489066 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.489071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.492682 (+ 3611us) server_negotiation.cc:407] Connection header received
0927 18:39:19.494021 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.494025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.494084 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.494137 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.495930 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.497868 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.499720 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.500080 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.500208 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.506650 (+ 6442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.506655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.506658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.506693 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.508475 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.510915 (+ 2440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.510919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.510921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.511016 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.511429 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.511433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.511434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.511667 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.511738 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.512551 (+ 813us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.512689 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:39:19.514761 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.489550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:46803 (local address 127.6.4.3:39615)
0927 18:39:19.492481 (+ 2931us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.492556 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.493716 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.494601 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.494612 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.494994 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:39:19.495739 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.495759 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.498104 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.498108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.499559 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.499574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.500392 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.500464 (+ 72us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.500698 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.504417 (+ 3719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:39:19.504446 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:39:19.506352 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:39:19.508669 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:19.508677 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:19.508701 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:19.510795 (+ 2094us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:19.511121 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:39:19.511124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:39:19.511126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:39:19.511316 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:39:19.511818 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:39:19.511827 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:39:19.512254 (+ 427us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.513265 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.513654 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2642,"thread_start_us":189,"threads_started":1}
I20260927 18:39:19.516606 17857 heartbeater.cc:347] Connected to a master server at 127.6.4.62:46803
I20260927 18:39:19.516886 17857 heartbeater.cc:464] Registering TS with master...
I20260927 18:39:19.517531 17857 heartbeater.cc:511] Master 127.6.4.62:46803 requested a full tablet report, sending...
I20260927 18:39:19.518883 17373 ts_manager.cc:194] Registered new tserver with Master: 34afbf33ec3641798a38393322bdfb95 (127.6.4.3:41093)
I20260927 18:39:19.519644 17373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:39615
I20260927 18:39:19.523262 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:39:19.535176 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.525767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.6.4.62:46803)
0927 18:39:19.525879 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.525883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.526369 (+ 486us) server_negotiation.cc:407] Connection header received
0927 18:39:19.526555 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.526558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.526604 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.526651 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:39:19.527579 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.528429 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.529632 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.529966 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.530034 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.531743 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:39:19.531747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:39:19.531750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:39:19.531776 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:39:19.533370 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.533853 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.533880 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.533882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.533966 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:39:19.534251 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:39:19.534254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:39:19.534256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:39:19.534468 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:39:19.534565 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.534897 (+ 332us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.535014 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:39:19.539146 17373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:39:19.543426 17373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:39:19.571973 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.557784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:37437 (local address 127.0.0.1:55772)
0927 18:39:19.561596 (+ 3812us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.561610 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.561683 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.567575 (+ 5892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.567578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.567595 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:19.567688 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.567693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.569041 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.569045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.570180 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.570190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.571423 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.571427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.571550 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.571590 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.571697 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.571815 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3753,"thread_start_us":84,"threads_started":1}
I20260927 18:39:19.573064 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.558369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:41093 (local address 127.0.0.1:37280)
0927 18:39:19.562553 (+ 4184us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.562565 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.562646 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.563668 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.563670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.563684 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:19.563749 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.563754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.570022 (+ 6268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.570025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.571352 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.571363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.572695 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.572699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.572789 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.572810 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.572874 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.572936 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4126,"thread_start_us":102,"threads_started":1}
I20260927 18:39:19.575814 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.555379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:33121 (local address 127.0.0.1:58168)
0927 18:39:19.560378 (+ 4999us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.560451 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.560579 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.563356 (+ 2777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.563363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.563386 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:19.563535 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.563541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.567157 (+ 3616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.567161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.568821 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.573286 (+ 4465us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.575390 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.575394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.575523 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.575546 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.575612 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.575677 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4806,"thread_start_us":93,"threads_started":1}
I20260927 18:39:19.576695 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.558745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.6.4.3:41093)
0927 18:39:19.560857 (+ 2112us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.560869 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.562736 (+ 1867us) server_negotiation.cc:407] Connection header received
0927 18:39:19.562819 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.562825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.563044 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.563190 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:19.563850 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.569901 (+ 6051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.571520 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.572392 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.572501 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.572636 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.576437 (+ 3801us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.576556 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2012,"thread_start_us":97,"threads_started":1}
I20260927 18:39:19.577065 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.557958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.6.4.1:37437)
0927 18:39:19.566924 (+ 8966us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.566933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.566959 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:39:19.567036 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.567043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.567259 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.567419 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:19.567806 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.568914 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.570310 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.571121 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.571215 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.571364 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.576814 (+ 5450us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.576927 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8807,"thread_start_us":83,"threads_started":1}
I20260927 18:39:19.577212 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.556251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.6.4.2:33121)
0927 18:39:19.560081 (+ 3830us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.560112 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.560910 (+ 798us) server_negotiation.cc:407] Connection header received
0927 18:39:19.560992 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.560998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.561222 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.561363 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:19.565782 (+ 4419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.567018 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.574220 (+ 7202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.575091 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.575196 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.575329 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.576929 (+ 1600us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.577083 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3711,"thread_start_us":115,"threads_started":1}
I20260927 18:39:19.580453 17790 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1ef2026c5f42aab67db9628bb744ac (DEFAULT_TABLE table=test-table [id=faa0de19dcea4ecfa90f508fa154b2cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:19.580453 17503 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1ef2026c5f42aab67db9628bb744ac (DEFAULT_TABLE table=test-table [id=faa0de19dcea4ecfa90f508fa154b2cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:19.581631 17503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1ef2026c5f42aab67db9628bb744ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:19.581631 17790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1ef2026c5f42aab67db9628bb744ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:19.593712 17877 tablet_bootstrap.cc:492] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Bootstrap starting.
I20260927 18:39:19.595656 17657 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1ef2026c5f42aab67db9628bb744ac (DEFAULT_TABLE table=test-table [id=faa0de19dcea4ecfa90f508fa154b2cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:39:19.596987 17657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1ef2026c5f42aab67db9628bb744ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:39:19.597314 17877 tablet_bootstrap.cc:654] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:19.598235 17877 log.cc:824] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:19.600389 17877 tablet_bootstrap.cc:492] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: No bootstrap required, opened a new log
I20260927 18:39:19.600582 17877 ts_tablet_manager.cc:1397] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260927 18:39:19.605815 17877 raft_consensus.cc:348] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.606321 17877 raft_consensus.cc:374] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:19.606415 17877 raft_consensus.cc:729] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34afbf33ec3641798a38393322bdfb95, State: Initialized, Role: FOLLOWER
I20260927 18:39:19.607026 17877 consensus_queue.cc:260] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.608211 17877 ts_tablet_manager.cc:1428] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260927 18:39:19.608611 17857 heartbeater.cc:503] Master 127.6.4.62:46803 was elected leader, sending a full tablet report...
I20260927 18:39:19.612515 17879 tablet_bootstrap.cc:492] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Bootstrap starting.
I20260927 18:39:19.613092 17876 tablet_bootstrap.cc:492] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: Bootstrap starting.
I20260927 18:39:19.615600 17879 tablet_bootstrap.cc:654] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:19.615978 17876 tablet_bootstrap.cc:654] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: Neither blocks nor log segments found. Creating new log.
I20260927 18:39:19.616905 17876 log.cc:824] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:19.621752 17879 log.cc:824] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:39:19.632211 17876 tablet_bootstrap.cc:492] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: No bootstrap required, opened a new log
I20260927 18:39:19.632300 17879 tablet_bootstrap.cc:492] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: No bootstrap required, opened a new log
I20260927 18:39:19.632570 17879 ts_tablet_manager.cc:1397] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260927 18:39:19.632720 17876 ts_tablet_manager.cc:1397] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.000s
I20260927 18:39:19.636883 17879 raft_consensus.cc:348] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.637212 17879 raft_consensus.cc:374] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:19.637277 17879 raft_consensus.cc:729] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb4caadc76143ebbba6d83e6efc90dd, State: Initialized, Role: FOLLOWER
I20260927 18:39:19.637043 17876 raft_consensus.cc:348] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.637542 17876 raft_consensus.cc:374] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:39:19.637637 17876 raft_consensus.cc:729] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc673964568c4e69b750c67b80866b61, State: Initialized, Role: FOLLOWER
I20260927 18:39:19.637873 17879 consensus_queue.cc:260] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.638187 17876 consensus_queue.cc:260] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.638957 17879 ts_tablet_manager.cc:1428] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 18:39:19.639235 17876 ts_tablet_manager.cc:1428] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 18:39:19.639627 17587 heartbeater.cc:503] Master 127.6.4.62:46803 was elected leader, sending a full tablet report...
I20260927 18:39:19.640013 17723 heartbeater.cc:503] Master 127.6.4.62:46803 was elected leader, sending a full tablet report...
Sep 27 18:39:19 dist-test-slave-bzzj krb5kdc[16897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:19.710376 17883 raft_consensus.cc:482] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:39:19.710596 17883 raft_consensus.cc:504] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.711721 17883 leader_election.cc:288] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc673964568c4e69b750c67b80866b61 (127.6.4.1:37437), 34afbf33ec3641798a38393322bdfb95 (127.6.4.3:41093)
I20260927 18:39:19.716945 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.712304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:35519 (local address 127.6.4.1:37437)
0927 18:39:19.712405 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.712410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.712425 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:39:19.712653 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.712656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.712707 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.712753 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:19.713114 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.713931 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.715576 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.716228 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.716309 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.716377 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.716732 (+ 355us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.716817 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:39:19.716952 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.711899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:37437 (local address 127.6.4.2:35519)
0927 18:39:19.712271 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.712286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.712504 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.712883 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.712886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.712899 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:19.713004 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.713010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.714074 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.714077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.715431 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.715443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.716503 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.716506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.716612 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.716637 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.716704 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.716783 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":23,"thread_start_us":101,"threads_started":1}
I20260927 18:39:19.717760 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.712190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:39591 (local address 127.6.4.3:41093)
0927 18:39:19.712320 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.712325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.712625 (+ 300us) server_negotiation.cc:407] Connection header received
0927 18:39:19.712785 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.712788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.712835 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.712881 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:39:19.713218 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.713940 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.715571 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.716228 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.716309 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.716377 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.717556 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.717641 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:39:19.717941 17523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1ef2026c5f42aab67db9628bb744ac" candidate_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc673964568c4e69b750c67b80866b61" is_pre_election: true
I20260927 18:39:19.718107 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.711963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:41093 (local address 127.6.4.2:39591)
0927 18:39:19.712556 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.712567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:39:19.712680 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:39:19.712994 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:39:19.712996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:39:19.713021 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:39:19.713108 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.713113 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.714076 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.714078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.715432 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:39:19.715443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.716732 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.716735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:39:19.716842 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.716863 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:39:19.717898 (+ 1035us) client_negotiation.cc:240] Negotiation successful
0927 18:39:19.717976 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":68,"thread_start_us":86,"threads_started":1}
I20260927 18:39:19.718266 17523 raft_consensus.cc:2454] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fb4caadc76143ebbba6d83e6efc90dd in term 0.
I20260927 18:39:19.718698 17795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1ef2026c5f42aab67db9628bb744ac" candidate_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34afbf33ec3641798a38393322bdfb95" is_pre_election: true
I20260927 18:39:19.718784 17613 leader_election.cc:302] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fb4caadc76143ebbba6d83e6efc90dd, fc673964568c4e69b750c67b80866b61; no voters:
I20260927 18:39:19.718997 17795 raft_consensus.cc:2454] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fb4caadc76143ebbba6d83e6efc90dd in term 0.
I20260927 18:39:19.719261 17883 raft_consensus.cc:2781] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:39:19.719347 17883 raft_consensus.cc:482] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:39:19.719388 17883 raft_consensus.cc:3037] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:19.720719 17883 raft_consensus.cc:504] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.721097 17883 leader_election.cc:288] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [CANDIDATE]: Term 1 election: Requested vote from peers fc673964568c4e69b750c67b80866b61 (127.6.4.1:37437), 34afbf33ec3641798a38393322bdfb95 (127.6.4.3:41093)
I20260927 18:39:19.721426 17795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1ef2026c5f42aab67db9628bb744ac" candidate_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34afbf33ec3641798a38393322bdfb95"
I20260927 18:39:19.721534 17795 raft_consensus.cc:3037] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:19.721565 17523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1ef2026c5f42aab67db9628bb744ac" candidate_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc673964568c4e69b750c67b80866b61"
I20260927 18:39:19.721678 17523 raft_consensus.cc:3037] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:39:19.722709 17795 raft_consensus.cc:2454] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fb4caadc76143ebbba6d83e6efc90dd in term 1.
I20260927 18:39:19.722867 17523 raft_consensus.cc:2454] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fb4caadc76143ebbba6d83e6efc90dd in term 1.
I20260927 18:39:19.723083 17610 leader_election.cc:302] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34afbf33ec3641798a38393322bdfb95, 8fb4caadc76143ebbba6d83e6efc90dd; no voters:
I20260927 18:39:19.723239 17883 raft_consensus.cc:2781] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:39:19.723450 17883 raft_consensus.cc:686] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 1 LEADER]: Becoming Leader. State: Replica: 8fb4caadc76143ebbba6d83e6efc90dd, State: Running, Role: LEADER
I20260927 18:39:19.723803 17883 consensus_queue.cc:237] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } }
I20260927 18:39:19.727368 17372 catalog_manager.cc:5495] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fb4caadc76143ebbba6d83e6efc90dd (127.6.4.2). New cstate: current_term: 1 leader_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fb4caadc76143ebbba6d83e6efc90dd" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 33121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 } health_report { overall_health: UNKNOWN } } }
W20260927 18:39:19.741526 17859 tablet.cc:2369] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:19.767529 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.762277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.6.4.2:33121)
0927 18:39:19.762605 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0927 18:39:19.762609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:39:19.762622 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:39:19.762702 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:39:19.762705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:39:19.762769 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:39:19.762865 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:39:19.763240 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.764084 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.765471 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:39:19.765819 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:39:19.765911 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:39:19.766322 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:39:19.766869 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:39:19.766981 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 18:39:19.767314 (+ 333us) server_negotiation.cc:299] Negotiation successful
0927 18:39:19.767385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260927 18:39:19.775255 17795 raft_consensus.cc:1261] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 1 FOLLOWER]: Refusing update from remote peer 8fb4caadc76143ebbba6d83e6efc90dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:19.776213 17883 consensus_queue.cc:1035] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "34afbf33ec3641798a38393322bdfb95" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:39:19.781930 17523 raft_consensus.cc:1261] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61 [term 1 FOLLOWER]: Refusing update from remote peer 8fb4caadc76143ebbba6d83e6efc90dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:39:19.782725 17883 consensus_queue.cc:1035] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc673964568c4e69b750c67b80866b61" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 37437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:39:19.800034 17588 tablet.cc:2369] T 7e1ef2026c5f42aab67db9628bb744ac P fc673964568c4e69b750c67b80866b61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:39:19.808187 17892 mvcc.cc:205] Tried to move back new op lower bound from 7334028737629761536 to 7334028737432358912. Current Snapshot: MvccSnapshot[applied={T|T < 7334028737629761536 or (T in {7334028737629761536})}]
I20260927 18:39:19.813508 17372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:19.813686 17372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:39:19.814193 17895 mvcc.cc:205] Tried to move back new op lower bound from 7334028737629761536 to 7334028737432358912. Current Snapshot: MvccSnapshot[applied={T|T < 7334028737629761536}]
I20260927 18:39:19.816344 17372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f9a2473d3584770912bc77f3b701ea2: Sending DeleteTablet for 3 replicas of tablet 7e1ef2026c5f42aab67db9628bb744ac
I20260927 18:39:19.817328 17503 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1ef2026c5f42aab67db9628bb744ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:19 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:55772
I20260927 18:39:19.817711 17657 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1ef2026c5f42aab67db9628bb744ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:19 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:58168
I20260927 18:39:19.818152 17790 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1ef2026c5f42aab67db9628bb744ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:39:19 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:37280
I20260927 18:39:19.818209 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 17454
I20260927 18:39:19.825889 17893 mvcc.cc:205] Tried to move back new op lower bound from 7334028737629761536 to 7334028737432358912. Current Snapshot: MvccSnapshot[applied={T|T < 7334028737629761536}]
I20260927 18:39:19.826644 17902 tablet_replica.cc:330] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: stopping tablet replica
I20260927 18:39:19.827049 17901 tablet_replica.cc:330] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: stopping tablet replica
I20260927 18:39:19.827440 17901 raft_consensus.cc:2229] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:39:19.827821 17902 raft_consensus.cc:2229] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:39:19.829686 17902 raft_consensus.cc:2258] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:19.830118 17901 raft_consensus.cc:2258] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:39:19.833181 17901 ts_tablet_manager.cc:1905] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:19.834589 17902 ts_tablet_manager.cc:1905] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:39:19.842902 17902 ts_tablet_manager.cc:1918] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:19.845775 17901 ts_tablet_manager.cc:1918] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:39:19.845981 17902 log.cc:1196] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/7e1ef2026c5f42aab67db9628bb744ac
I20260927 18:39:19.846242 17901 log.cc:1196] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/7e1ef2026c5f42aab67db9628bb744ac
I20260927 18:39:19.846668 17902 ts_tablet_manager.cc:1939] T 7e1ef2026c5f42aab67db9628bb744ac P 34afbf33ec3641798a38393322bdfb95: Deleting consensus metadata
I20260927 18:39:19.847014 17901 ts_tablet_manager.cc:1939] T 7e1ef2026c5f42aab67db9628bb744ac P 8fb4caadc76143ebbba6d83e6efc90dd: Deleting consensus metadata
I20260927 18:39:19.848532 17358 catalog_manager.cc:4841] TS 8fb4caadc76143ebbba6d83e6efc90dd (127.6.4.2:33121): tablet 7e1ef2026c5f42aab67db9628bb744ac (table test-table [id=faa0de19dcea4ecfa90f508fa154b2cd]) successfully deleted
I20260927 18:39:19.850097 17357 catalog_manager.cc:4841] TS 34afbf33ec3641798a38393322bdfb95 (127.6.4.3:41093): tablet 7e1ef2026c5f42aab67db9628bb744ac (table test-table [id=faa0de19dcea4ecfa90f508fa154b2cd]) successfully deleted
W20260927 18:39:19.851075 17360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:37437 (error 108)
I20260927 18:39:19.852599 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 17593
I20260927 18:39:19.852643 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:39:19.852335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:37437 (local address 127.0.0.1:55776)
0927 18:39:19.852477 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:39:19.852555 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:37437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:39:19.853263 17360 catalog_manager.cc:4584] TS fc673964568c4e69b750c67b80866b61 (127.6.4.1:37437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e1ef2026c5f42aab67db9628bb744ac: Network error: Client connection negotiation failed: client connection to 127.6.4.1:37437: connect: Connection refused (error 111)
I20260927 18:39:19.862080 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 17728
I20260927 18:39:19.870262 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 17341
I20260927 18:39:19.877597 6160 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 18:39:20.212685 6160 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 18:39:20.212815 6160 mini_ranger.cc:66] Stopping Ranger...
I20260927 18:39:20.547927 6160 mini_ranger.cc:68] Stopped Ranger
2026-09-27 18:39:20.548 UTC [16927] LOG: received smart shutdown request
2026-09-27 18:39:20.551 UTC [16927] LOG: background worker "logical replication launcher" (PID 16936) exited with exit code 1
2026-09-27 18:39:20.551 UTC [16931] LOG: shutting down
2026-09-27 18:39:20.706 UTC [16927] LOG: database system is shut down
2026-09-27T18:39:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (42836 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:39:20 dist-test-slave-bzzj krb5kdc[17913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:39:20 dist-test-slave-bzzj krb5kdc[17913](info): set up 2 sockets
Sep 27 18:39:20 dist-test-slave-bzzj krb5kdc[17913](info): commencing operation
krb5kdc: starting...
W20260927 18:39:21.780390 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:39:21 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:39:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:39:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/rangeradmin_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/rangeradmin_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/rangerlookup_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/rangerlookup_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
I20260927 18:39:21.941087 6160 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFEZSEE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/postgres -l logfile start
2026-09-27 18:39:24.230 UTC [17943] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 18:39:24.230 UTC [17943] LOG: listening on IPv4 address "127.6.4.20", port 59869
2026-09-27 18:39:24.233 UTC [17943] LOG: listening on Unix socket "/tmp/.s.PGSQL.59869"
2026-09-27 18:39:24.259 UTC [17946] LOG: database system was shut down at 2026-09-27 18:39:22 UTC
2026-09-27 18:39:24.263 UTC [17943] LOG: database system is ready to accept connections
I20260927 18:39:25.119554 6160 mini_postgres.cc:96] Postgres bound to 59869
2026-09-27 18:39:25.125 UTC [17954] FATAL: database "slave" does not exist
127.6.4.20:59869 - accepting connections
I20260927 18:39:25.125799 6160 mini_ranger.cc:161] Starting Ranger...
I20260927 18:39:25.139426 6160 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 18:39:25.539377 6160 mini_ranger.cc:87] Created ranger Postgres database
I20260927 18:39:25.539465 6160 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-admin
2026-09-27 18:39:25,809 [I] DB FLAVOR :POSTGRES
2026-09-27 18:39:25,809 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:39:25,810 [I] Checking connection..
2026-09-27 18:39:25,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 18:39:26,040 [I] Checking connection passed.
2026-09-27 18:39:26,040 [I] --------- Verifying version history table ---------
2026-09-27 18:39:26,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:39:26,254 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:39:26,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:39:26,494 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:39:26,494 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 18:39:26,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 18:39:26.704 UTC [18018] WARNING: there is no transaction in progress
2026-09-27 18:39:26,709 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 18:39:26,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:39:26,946 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 18:39:26,946 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:39:26,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 18:39:27,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 18:39:27,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bzzj.c.gcp-upstream.internal','N') ;"
2026-09-27 18:39:27,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:39:27,885 [I] Table x_portal_user does not exist in database ranger
2026-09-27 18:39:27,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:39:28,123 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 18:39:28,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:39:28,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:39:28,618 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 18:39:28,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 18:39:29.495 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.510 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.524 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.538 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.552 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.567 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.616 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.622 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.634 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.643 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.652 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.664 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.672 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.678 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.683 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:29.689 UTC [18126] WARNING: there is no transaction in progress
2026-09-27 18:39:30,231 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 18:39:30,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 18:39:30,464 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 18:39:30,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:39:30,681 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 18:39:30,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:39:30,925 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 18:39:30,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:39:31,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:39:31,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bzzj.c.gcp-upstream.internal';"
2026-09-27 18:39:31,597 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 18:39:31,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 18:39:31,841 [I] DB_PATCHES have already been applied
I20260927 18:39:31.846463 6160 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFEZSEE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 18:39:31.846521 6160 mini_ranger.cc:193] Using host: 127.6.4.20
I20260927 18:39:31.849659 6160 mini_ranger.cc:227] Ranger admin URL: http://127.6.4.20:48441
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 6:39:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 18:39:32.840776 6160 mini_ranger.cc:160] Time spent starting Ranger: real 7.715s user 0.000s sys 0.008s
Sep 27, 2026 6:39:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 6:39:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 6:39:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 6:39:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 6:39:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 18:39:53.825205 6160 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.4.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.4.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/rangerkms_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.4.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/rangerkms_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.4.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.4.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.4.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/HTTP_127.6.4.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 18:39:53 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534393, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 18:39:53.905570 6160 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 18:39:53.913304 6160 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 18:39:54.656698 6160 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 18:39:54.656782 6160 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms
I20260927 18:39:54.656800 6160 mini_ranger_kms.cc:227] Using postgres at 127.6.4.20:59869
2026-09-27 18:39:54,806 [I] DB FLAVOR :POSTGRES
2026-09-27 18:39:54,806 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:39:54,806 [I] Checking connection
2026-09-27 18:39:54,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 18:39:55,065 [I] connection success
2026-09-27 18:39:55,065 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 18:39:55,065 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 18:39:55,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 18:39:55,376 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 18:39:55,376 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:39:55,376 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 18:39:55,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.4.20:59869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 18:39:55,712 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 18:39:56.142495 6160 mini_ranger_kms.cc:310] Created kms service
I20260927 18:39:56.314131 6160 mini_ranger_kms.cc:326] Created kudu user
I20260927 18:39:56.356983 6160 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 18:39:56 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534393, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:39:56.725651 6160 mini_ranger_kms.cc:384] Added ranger policy
I20260927 18:39:56.725790 6160 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFEZSEE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFEZSEE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 18:39:56.725809 6160 mini_ranger_kms.cc:240] Using host: 127.6.4.20
I20260927 18:39:56.729306 6160 mini_ranger_kms.cc:276] Ranger KMS PID: 18306
I20260927 18:39:56.729389 6160 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.4.20:57685
Sep 27, 2026 6:39:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:39:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:39:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 6:39:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 6:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 6:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 18:39:58.799194 6160 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.894s user 0.014s sys 0.120s
I20260927 18:39:58.799317 6160 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 18:39:58.799397 6160 mini_ranger_kms.cc:401] 127.6.4.20:57685/kms/v1/keys
Sep 27 18:39:58 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534398, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.4.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 18:39:59 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534398, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.4.20@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
Sep 27 18:39:59 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534398, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.4.20@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
Sep 27 18:40:01 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:40:01.810923 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42599
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42599
--ranger_config_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:01.941347 18357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:01.941623 18357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:01.941680 18357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:01.945739 18357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:01.945808 18357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:01.945833 18357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:01.945855 18357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:01.945884 18357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:01.951537 18357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.4.20:57685/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42599
--ranger_config_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:01.952652 18357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:01.953665 18357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:01.963321 18362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:01.963346 18363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:01.963459 18357 server_base.cc:1034] running on GCE node
W20260927 18:40:01.963636 18365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:01.964304 18357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:01.965758 18357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:01.966962 18357 hybrid_clock.cc:648] HybridClock initialized: now 1790534401966935 us; error 44 us; skew 500 ppm
Sep 27 18:40:01 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534401, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:01.970494 18357 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:01.971480 18357 webserver.cc:460] Webserver started at http://127.6.4.62:43689/ using document root <none> and password file <none>
I20260927 18:40:01.972019 18357 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:01.972070 18357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:01.972287 18357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:01 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534401, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:02.033427 18357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "2fda5b30931047bcb83f876e355546e1"
format_stamp: "Formatted at 2026-09-27 18:40:02 on dist-test-slave-bzzj"
server_key: "59b0c39fee57312a3f14b7a658d9edac"
server_key_iv: "aedd31a84971edd4969d772c84b04a5c"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:02.034190 18357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "2fda5b30931047bcb83f876e355546e1"
format_stamp: "Formatted at 2026-09-27 18:40:02 on dist-test-slave-bzzj"
server_key: "59b0c39fee57312a3f14b7a658d9edac"
server_key_iv: "aedd31a84971edd4969d772c84b04a5c"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:02.039142 18357 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 18:40:02.056326 18372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:02.057971 18357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:40:02.058125 18357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "2fda5b30931047bcb83f876e355546e1"
format_stamp: "Formatted at 2026-09-27 18:40:02 on dist-test-slave-bzzj"
server_key: "59b0c39fee57312a3f14b7a658d9edac"
server_key_iv: "aedd31a84971edd4969d772c84b04a5c"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:02.058264 18357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:02.074302 18357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:02.077456 18357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:02.077638 18357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:02.085691 18357 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:42599
I20260927 18:40:02.085692 18424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:42599 every 8 connection(s)
I20260927 18:40:02.086877 18357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:02.089103 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 18357
I20260927 18:40:02.089226 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:40:02.089535 6160 external_mini_cluster.cc:1428] Setting key 739ae9b5c47d1b00153e9d8c72f3c786
I20260927 18:40:02.089991 18425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:02.096866 18425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Bootstrap starting.
I20260927 18:40:02.099524 18425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Neither blocks nor log segments found. Creating new log.
Sep 27 18:40:02 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:02.100327 18425 log.cc:824] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:02.102576 18425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: No bootstrap required, opened a new log
I20260927 18:40:02.105770 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:02.090969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.6.4.62:42599)
0927 18:40:02.091430 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:02.091449 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:02.091493 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:40:02.092424 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:02.092448 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:02.092881 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:02.093257 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:02.094161 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:02.095730 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:02.096955 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:02.097717 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:02.097865 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:02.100857 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:02.100864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:02.100884 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:02.100913 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:02.103123 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:02.103525 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:02.103530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:02.103535 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:02.103636 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:02.103888 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:02.103891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:02.103893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:02.104328 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:02.104533 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:02.104798 (+ 265us) server_negotiation.cc:299] Negotiation successful
0927 18:40:02.105062 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":135,"threads_started":1}
I20260927 18:40:02.106639 18425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:02.106994 18425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:02.107069 18425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fda5b30931047bcb83f876e355546e1, State: Initialized, Role: FOLLOWER
I20260927 18:40:02.107578 18425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:02.107733 18425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:02.107815 18425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:02.107909 18425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:02.109052 18425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:02.109407 18425 leader_election.cc:302] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fda5b30931047bcb83f876e355546e1; no voters:
I20260927 18:40:02.109728 18425 leader_election.cc:288] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:40:02.109814 18430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:02.110030 18430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 LEADER]: Becoming Leader. State: Replica: 2fda5b30931047bcb83f876e355546e1, State: Running, Role: LEADER
I20260927 18:40:02.110378 18430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:02.110965 18425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:02.112269 18432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fda5b30931047bcb83f876e355546e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } } }
I20260927 18:40:02.112385 18432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:02.112612 18432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fda5b30931047bcb83f876e355546e1. Latest consensus state: current_term: 1 leader_uuid: "2fda5b30931047bcb83f876e355546e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } } }
I20260927 18:40:02.112681 18432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:02.113102 18425 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:40:02.116492 18441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:02.120265 18441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:02.126839 18441 catalog_manager.cc:1347] Generated new cluster ID: 6062c252450948ebb96a3c3ca46d629c
I20260927 18:40:02.126991 18441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:02.168880 18441 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:40:02.170379 18441 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:02.180133 18441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Generated new TSK 0
I20260927 18:40:02.181245 18441 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:40:02.878619 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42599
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:03.028497 18470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:03.028860 18470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:03.028925 18470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:03.034926 18470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:03.034991 18470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:03.035053 18470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:03.041260 18470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.4.20:57685/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:03.042712 18470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:03.044085 18470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:03.056178 18480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:03.056205 18477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:03.056211 18478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:03.056679 18470 server_base.cc:1034] running on GCE node
I20260927 18:40:03.057165 18470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:03.057976 18470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:03.059182 18470 hybrid_clock.cc:648] HybridClock initialized: now 1790534403059148 us; error 41 us; skew 500 ppm
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:03.062601 18470 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:03.063716 18470 webserver.cc:460] Webserver started at http://127.6.4.1:39319/ using document root <none> and password file <none>
I20260927 18:40:03.064545 18470 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:03.064637 18470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:03.064929 18470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:03.106000 18470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "57231e9cce1b4c39982d1eedf9a72a88"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "67a8a7d2fd3a5e25fd0a5675853e1565"
server_key_iv: "69aa27d1c596dc1d585de055db339cc3"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.106684 18470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "57231e9cce1b4c39982d1eedf9a72a88"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "67a8a7d2fd3a5e25fd0a5675853e1565"
server_key_iv: "69aa27d1c596dc1d585de055db339cc3"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.111549 18470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260927 18:40:03.142549 18489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.144404 18470 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 18:40:03.144555 18470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "57231e9cce1b4c39982d1eedf9a72a88"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "67a8a7d2fd3a5e25fd0a5675853e1565"
server_key_iv: "69aa27d1c596dc1d585de055db339cc3"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.144701 18470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:03.181324 18470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:03.185166 18470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:03.185364 18470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:03.186102 18470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:03.187170 18470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:03.187232 18470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.187288 18470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:03.187347 18470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 18:40:03.201241 18470 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:41743
I20260927 18:40:03.202527 18470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:03.207917 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 18470
I20260927 18:40:03.208045 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:40:03.204561 18602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:41743 every 8 connection(s)
I20260927 18:40:03.208379 6160 external_mini_cluster.cc:1428] Setting key 4d828df8d710740fd7207c5faf143f4f
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:03.228312 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:03.207209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:34919 (local address 127.6.4.62:42599)
0927 18:40:03.207458 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:03.207462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:03.207479 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:40:03.207986 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:03.207990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:03.208049 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:03.208221 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:03.216196 (+ 7975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.217087 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.218895 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.219229 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.219314 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:03.223913 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:03.223916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:03.223920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:03.223949 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:03.225729 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:03.226383 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:03.226387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:03.226388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:03.226488 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:03.226935 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:03.226938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:03.226940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:03.227203 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:03.227297 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:03.228024 (+ 727us) server_negotiation.cc:299] Negotiation successful
0927 18:40:03.228171 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260927 18:40:03.229380 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:03.206011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42599 (local address 127.6.4.1:34919)
0927 18:40:03.206644 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:03.206709 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:03.207723 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:03.214760 (+ 7037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:03.214771 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:03.215174 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:03.215963 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:03.215988 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.217263 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.217266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:03.218749 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:03.218762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.219487 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.219551 (+ 64us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:03.219766 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:03.220609 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:03.220628 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:03.223701 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:03.225868 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:03.225875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:03.225893 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:03.226269 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:03.226595 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:03.226599 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:03.226601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:03.226808 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:03.227382 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:03.227389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:03.227741 (+ 352us) client_negotiation.cc:769] Sending connection context
0927 18:40:03.227953 (+ 212us) client_negotiation.cc:240] Negotiation successful
0927 18:40:03.228348 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":145,"threads_started":1}
I20260927 18:40:03.230719 18603 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42599
I20260927 18:40:03.231009 18603 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:03.231637 18603 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:03.233449 18389 ts_manager.cc:194] Registered new tserver with Master: 57231e9cce1b4c39982d1eedf9a72a88 (127.6.4.1:41743)
I20260927 18:40:03.235006 18389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:34919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:40:03.240597 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42599
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:03.404211 18609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:03.404613 18609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:03.404750 18609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:03.410830 18609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:03.410961 18609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:03.411084 18609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:03.419092 18609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.4.20:57685/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:03.420763 18609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:03.422286 18609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:03.435560 18615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:03.435554 18616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:03.435983 18618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:03.436342 18609 server_base.cc:1034] running on GCE node
I20260927 18:40:03.436959 18609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:03.437773 18609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:03.438968 18609 hybrid_clock.cc:648] HybridClock initialized: now 1790534403438940 us; error 39 us; skew 500 ppm
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:03.442339 18609 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:03.443368 18609 webserver.cc:460] Webserver started at http://127.6.4.2:46337/ using document root <none> and password file <none>
I20260927 18:40:03.443918 18609 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:03.443982 18609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:03.444222 18609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:03.459970 18609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "2bbe36fe654f4e06a65d66f095287af2"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "afc1c2712a7158eae5eb4302e549453f"
server_key_iv: "33618e5c93592514522d8ab7f78718c5"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.460775 18609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "2bbe36fe654f4e06a65d66f095287af2"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "afc1c2712a7158eae5eb4302e549453f"
server_key_iv: "33618e5c93592514522d8ab7f78718c5"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.465960 18609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 18:40:03.492702 18625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.494592 18609 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260927 18:40:03.494733 18609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "2bbe36fe654f4e06a65d66f095287af2"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "afc1c2712a7158eae5eb4302e549453f"
server_key_iv: "33618e5c93592514522d8ab7f78718c5"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.494874 18609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:03.511834 18609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:03.515242 18609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:03.515424 18609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:03.516021 18609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:03.516916 18609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:03.516966 18609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.517019 18609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:03.517046 18609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.526271 18609 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36531
I20260927 18:40:03.527513 18609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:40:03.531040 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 18609
I20260927 18:40:03.531154 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:40:03.531433 6160 external_mini_cluster.cc:1428] Setting key 85ebe85b005b72c0cfc16928cf636f15
I20260927 18:40:03.537667 18738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36531 every 8 connection(s)
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:03.559283 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:03.542224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:49295 (local address 127.6.4.62:42599)
0927 18:40:03.542372 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:03.542377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:03.543600 (+ 1223us) server_negotiation.cc:407] Connection header received
0927 18:40:03.544964 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:03.544968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:03.545027 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:03.545086 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:03.546797 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.547635 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.549367 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.549767 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.549884 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:03.553379 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:03.553383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:03.553386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:03.553429 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:03.556020 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:03.556848 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:03.556852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:03.556854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:03.556998 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:03.557485 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:03.557489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:03.557491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:03.557741 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:03.557840 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:03.558953 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0927 18:40:03.559099 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:40:03.560349 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:03.542775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42599 (local address 127.6.4.2:49295)
0927 18:40:03.543428 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:03.543492 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:03.544675 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:03.545541 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:03.545551 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:03.545926 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:03.546615 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:03.546637 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.547794 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.547797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:03.549223 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:03.549234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.550035 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.550097 (+ 62us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:03.550306 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:03.551212 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:03.551235 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:03.553104 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:03.556269 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:03.556275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:03.556296 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:03.556659 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:03.557112 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:03.557115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:03.557117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:03.557322 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:03.558230 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:03.558238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:03.558630 (+ 392us) client_negotiation.cc:769] Sending connection context
0927 18:40:03.558855 (+ 225us) client_negotiation.cc:240] Negotiation successful
0927 18:40:03.559228 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":179,"threads_started":1}
I20260927 18:40:03.561865 18739 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42599
I20260927 18:40:03.562135 18739 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:03.562731 18739 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:03.564070 18389 ts_manager.cc:194] Registered new tserver with Master: 2bbe36fe654f4e06a65d66f095287af2 (127.6.4.2:36531)
I20260927 18:40:03.564885 18389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:49295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:40:03.568333 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42599
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
W20260927 18:40:03.725235 18744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:03.725631 18744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:03.725724 18744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:03.731829 18744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:03.731949 18744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:03.732057 18744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:03.739806 18744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.4.20:57685/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:03.741650 18744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:03.742944 18744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:03.755369 18750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:03.755631 18744 server_base.cc:1034] running on GCE node
W20260927 18:40:03.755869 18752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:03.760588 18749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:03.761176 18744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:03.762032 18744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:03.763208 18744 hybrid_clock.cc:648] HybridClock initialized: now 1790534403763173 us; error 53 us; skew 500 ppm
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:03.766940 18744 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:03.768124 18744 webserver.cc:460] Webserver started at http://127.6.4.3:43423/ using document root <none> and password file <none>
I20260927 18:40:03.768862 18744 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:03.768949 18744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:03.769182 18744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:03.795516 18744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "04b0a49f371e4073832232fb55850d9c"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "0c2cb66d04fe1be8cdc71fe83a35eced"
server_key_iv: "b06e10c927459408b8b508fcaaaf48cb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.796407 18744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "04b0a49f371e4073832232fb55850d9c"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "0c2cb66d04fe1be8cdc71fe83a35eced"
server_key_iv: "b06e10c927459408b8b508fcaaaf48cb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.800863 18744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 18:40:03.821030 18759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.822834 18744 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260927 18:40:03.823045 18744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "04b0a49f371e4073832232fb55850d9c"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "0c2cb66d04fe1be8cdc71fe83a35eced"
server_key_iv: "b06e10c927459408b8b508fcaaaf48cb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:03.823282 18744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:03.855372 18744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:03.858331 18744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:03.858510 18744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:03.859113 18744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:03.860136 18744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:03.860191 18744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.860242 18744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:03.860265 18744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:03.870541 18744 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:44723
I20260927 18:40:03.870676 18872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:44723 every 8 connection(s)
I20260927 18:40:03.871472 18744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:03.872293 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 18744
I20260927 18:40:03.872417 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:40:03.872718 6160 external_mini_cluster.cc:1428] Setting key 26069c472ed431c2e7ed35c2101fc6c7
Sep 27 18:40:03 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534403, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:03.885269 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:03.873531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:49213 (local address 127.6.4.62:42599)
0927 18:40:03.873661 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:03.873665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:03.875418 (+ 1753us) server_negotiation.cc:407] Connection header received
0927 18:40:03.876404 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:03.876408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:03.876464 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:03.876518 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:03.877763 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.878293 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.879473 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.879691 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.879776 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:03.882038 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:03.882041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:03.882043 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:03.882067 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:03.883316 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:03.883782 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:03.883786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:03.883787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:03.883859 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:03.884211 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:03.884214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:03.884216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:03.884390 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:03.884475 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:03.884991 (+ 516us) server_negotiation.cc:299] Negotiation successful
0927 18:40:03.885108 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:40:03.886529 18875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:03.874019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42599 (local address 127.6.4.3:49213)
0927 18:40:03.875283 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:03.875327 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:03.876164 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:03.876867 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:03.876873 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:03.877140 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:03.877626 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:03.877639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.878408 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.878411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:03.879354 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:03.879363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:03.879772 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:03.879824 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:03.879997 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:03.880584 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:03.880601 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:03.881881 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:03.883421 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:03.883427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:03.883443 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:03.883690 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:03.883938 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:03.883941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:03.883944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:03.884133 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:03.884494 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:03.884501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:03.884752 (+ 251us) client_negotiation.cc:769] Sending connection context
0927 18:40:03.884909 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 18:40:03.885497 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1063,"thread_start_us":138,"threads_started":1}
I20260927 18:40:03.887838 18873 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42599
I20260927 18:40:03.888068 18873 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:03.888563 18873 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:03.889492 18389 ts_manager.cc:194] Registered new tserver with Master: 04b0a49f371e4073832232fb55850d9c (127.6.4.3:44723)
I20260927 18:40:03.890027 18389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:49213
I20260927 18:40:03.899425 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:03.900292 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 18470
I20260927 18:40:03.906515 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 18609
I20260927 18:40:03.913489 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 18744
I20260927 18:40:03.918777 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 18357
I20260927 18:40:03.924152 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42599
--webserver_interface=127.6.4.62
--webserver_port=43689
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42599
--ranger_config_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:04.031222 18876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:04.031502 18876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:04.031548 18876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:04.035543 18876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:04.035605 18876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:04.035624 18876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:04.035640 18876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:04.035653 18876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:04.041157 18876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42599
--ranger_config_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=43689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:04.042173 18876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:04.043150 18876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:04.051419 18882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:04.051501 18876 server_base.cc:1034] running on GCE node
W20260927 18:40:04.051433 18881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:04.051433 18884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:04.052220 18876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:04.053334 18876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:04.054483 18876 hybrid_clock.cc:648] HybridClock initialized: now 1790534404054466 us; error 31 us; skew 500 ppm
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:04.057252 18876 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:04.058122 18876 webserver.cc:460] Webserver started at http://127.6.4.62:43689/ using document root <none> and password file <none>
I20260927 18:40:04.058636 18876 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:04.058698 18876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:04.120736 18876 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:40:04.123018 18892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.124334 18876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:04.124454 18876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "2fda5b30931047bcb83f876e355546e1"
format_stamp: "Formatted at 2026-09-27 18:40:02 on dist-test-slave-bzzj"
server_key: "59b0c39fee57312a3f14b7a658d9edac"
server_key_iv: "aedd31a84971edd4969d772c84b04a5c"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:04.124997 18876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:04.150741 18876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:04.153741 18876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:04.153918 18876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:04.160940 18876 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:42599
I20260927 18:40:04.160955 18944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:42599 every 8 connection(s)
I20260927 18:40:04.162087 18876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:04.166421 18945 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:40:04.168118 18945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Bootstrap starting.
I20260927 18:40:04.171638 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 18876
I20260927 18:40:04.172075 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:41743
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:42599
--webserver_port=39319
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:04.184448 18945 log.cc:824] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:04.191296 18945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:04.191675 18945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Bootstrap complete.
I20260927 18:40:04.195679 18945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:04.196086 18945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fda5b30931047bcb83f876e355546e1, State: Initialized, Role: FOLLOWER
I20260927 18:40:04.196633 18945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:04.196758 18945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:04.196825 18945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:04.196918 18945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:40:04.198796 18945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:04.199126 18945 leader_election.cc:302] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fda5b30931047bcb83f876e355546e1; no voters:
I20260927 18:40:04.199415 18945 leader_election.cc:288] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:40:04.199558 18949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:40:04.199813 18949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [term 2 LEADER]: Becoming Leader. State: Replica: 2fda5b30931047bcb83f876e355546e1, State: Running, Role: LEADER
I20260927 18:40:04.200213 18949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } }
I20260927 18:40:04.200656 18945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:04.201730 18951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fda5b30931047bcb83f876e355546e1. Latest consensus state: current_term: 2 leader_uuid: "2fda5b30931047bcb83f876e355546e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } } }
I20260927 18:40:04.201735 18950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2fda5b30931047bcb83f876e355546e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda5b30931047bcb83f876e355546e1" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42599 } } }
I20260927 18:40:04.201856 18951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:04.201864 18950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:04.202298 18958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:04.202811 18945 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:40:04.208354 18958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:04.209128 18958 catalog_manager.cc:1259] Loaded cluster ID: 6062c252450948ebb96a3c3ca46d629c
I20260927 18:40:04.209225 18958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:04.211763 18958 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:04.212666 18958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Loaded TSK: 0
I20260927 18:40:04.214025 18958 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:40:04.291373 18947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:04.291642 18947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:04.291702 18947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:04.296178 18947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:04.296250 18947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:04.296331 18947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:04.302117 18947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:41743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=39319
--tserver_master_addrs=127.6.4.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:04.303571 18947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:04.304968 18947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:04.317552 18983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:04.317569 18985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:04.317934 18982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:04.318046 18947 server_base.cc:1034] running on GCE node
I20260927 18:40:04.318580 18947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:04.319416 18947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:04.320614 18947 hybrid_clock.cc:648] HybridClock initialized: now 1790534404320578 us; error 48 us; skew 500 ppm
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:04.324164 18947 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:04.325349 18947 webserver.cc:460] Webserver started at http://127.6.4.1:39319/ using document root <none> and password file <none>
I20260927 18:40:04.326112 18947 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:04.326225 18947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:04.358296 18947 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.000s sys 0.003s
I20260927 18:40:04.361646 18993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.363395 18947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 18:40:04.363585 18947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "57231e9cce1b4c39982d1eedf9a72a88"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "67a8a7d2fd3a5e25fd0a5675853e1565"
server_key_iv: "69aa27d1c596dc1d585de055db339cc3"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:04.364436 18947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:04.389021 18947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:04.392614 18947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:04.392803 18947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:04.393450 18947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:04.394438 18947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:04.394481 18947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.394529 18947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:04.394550 18947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.405046 18947 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:41743
I20260927 18:40:04.405297 19106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:41743 every 8 connection(s)
I20260927 18:40:04.407281 18947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:04.408914 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 18947
I20260927 18:40:04.409373 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:36531
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:42599
--webserver_port=46337
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:04.431905 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.409888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42599 (local address 127.6.4.1:37607)
0927 18:40:04.411453 (+ 1565us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:04.411509 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:04.412623 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:04.414447 (+ 1824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:04.414457 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:04.414820 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:04.415461 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:04.415484 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.417379 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.417382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:04.418758 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:04.418770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.420007 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.420067 (+ 60us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:04.420257 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.420895 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:04.420917 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:04.425790 (+ 4873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:04.428054 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:04.428060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:04.428077 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:04.428467 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:04.428873 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:04.428877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:04.428879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:04.429063 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:04.430057 (+ 994us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:04.430063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:04.430372 (+ 309us) client_negotiation.cc:769] Sending connection context
0927 18:40:04.430555 (+ 183us) client_negotiation.cc:240] Negotiation successful
0927 18:40:04.430885 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":147,"threads_started":1}
I20260927 18:40:04.432231 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.409756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:37607 (local address 127.6.4.62:42599)
0927 18:40:04.412681 (+ 2925us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:04.412704 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:04.412754 (+ 50us) server_negotiation.cc:407] Connection header received
0927 18:40:04.413089 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:04.413106 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:04.413548 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:04.413939 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:04.415668 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.417215 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.418895 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.419692 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.419848 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.426024 (+ 6176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:04.426031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:04.426048 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:04.426082 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:04.427861 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.428588 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.428596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.428602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.428753 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.429189 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.429193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.429194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.429808 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:04.429993 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:04.431203 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0927 18:40:04.431455 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2767,"thread_start_us":126,"threads_started":1}
I20260927 18:40:04.434823 19107 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42599
I20260927 18:40:04.435086 19107 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.435544 19107 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
W20260927 18:40:04.437704 19107 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:42599 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:40:04.437834 19107 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.438009 19107 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:04.438818 19107 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.438994 19107 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
W20260927 18:40:04.439718 19107 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:40:04.546473 19111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:04.546834 19111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:04.546900 19111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:04.552884 19111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:04.552951 19111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:04.553038 19111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:04.559252 19111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:36531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=46337
--tserver_master_addrs=127.6.4.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:04.560523 19111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:04.561543 19111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:04.573274 19119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:04.573292 19116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:04.574285 19111 server_base.cc:1034] running on GCE node
W20260927 18:40:04.576213 19117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:04.576709 19111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:04.577580 19111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:04.578779 19111 hybrid_clock.cc:648] HybridClock initialized: now 1790534404578705 us; error 81 us; skew 500 ppm
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:04.582448 19111 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:04.583622 19111 webserver.cc:460] Webserver started at http://127.6.4.2:46337/ using document root <none> and password file <none>
I20260927 18:40:04.584385 19111 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:04.584501 19111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:04.603968 19111 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 18:40:04.607139 19127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.608654 19111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:04.608795 19111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "2bbe36fe654f4e06a65d66f095287af2"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "afc1c2712a7158eae5eb4302e549453f"
server_key_iv: "33618e5c93592514522d8ab7f78718c5"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:04.609499 19111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:04.643778 19111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:04.647264 19111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:04.647493 19111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:04.648243 19111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:04.649235 19111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:04.649299 19111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.649369 19111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:04.649403 19111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.657804 19111 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36531
I20260927 18:40:04.658854 19111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:40:04.668565 19240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36531 every 8 connection(s)
I20260927 18:40:04.668984 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19111
I20260927 18:40:04.669389 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.4.20:57685/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:44723
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:42599
--webserver_port=43423
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:04.688630 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.675028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:59219 (local address 127.6.4.62:42599)
0927 18:40:04.675163 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:04.675168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:04.675645 (+ 477us) server_negotiation.cc:407] Connection header received
0927 18:40:04.676975 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:04.676980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:04.677045 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:04.677102 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:04.678855 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.679691 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.681523 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.681878 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.681953 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.684682 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:04.684685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:04.684688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:04.684720 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:04.686310 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.686891 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.686895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.686897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.686979 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.687340 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.687344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.687346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.687567 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:04.687643 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:04.688316 (+ 673us) server_negotiation.cc:299] Negotiation successful
0927 18:40:04.688446 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:40:04.690450 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.673788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42599 (local address 127.6.4.2:59219)
0927 18:40:04.675462 (+ 1674us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:04.675536 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:04.676699 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:04.677579 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:04.677589 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:04.677944 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:04.678667 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:04.678689 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.679860 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.679863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:04.681363 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:04.681374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.682100 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.682156 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:04.682348 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.682954 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:04.682975 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:04.684495 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:04.686431 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:04.686437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:04.686453 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:04.686782 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:04.687074 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:04.687077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:04.687079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:04.687240 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:04.687706 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:04.687713 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:04.688025 (+ 312us) client_negotiation.cc:769] Sending connection context
0927 18:40:04.689065 (+ 1040us) client_negotiation.cc:240] Negotiation successful
0927 18:40:04.689393 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"thread_start_us":163,"threads_started":1}
I20260927 18:40:04.692001 19241 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42599
I20260927 18:40:04.692250 19241 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.692729 19241 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
W20260927 18:40:04.694293 19241 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:42599 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:40:04.694407 19241 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.694608 19241 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:04.695389 19241 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.695561 19241 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
W20260927 18:40:04.696336 19241 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:40:04.800973 19243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:04.801337 19243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:04.801400 19243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:04.806082 19243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:04.806159 19243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:04.806239 19243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:04.813583 19243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.4.20:57685/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:44723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=43423
--tserver_master_addrs=127.6.4.62:42599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:04.814741 19243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:04.815798 19243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:04.825446 19250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:04.825446 19251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:04.825595 19253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:04.825898 19243 server_base.cc:1034] running on GCE node
I20260927 18:40:04.826282 19243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:04.826927 19243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:04.828116 19243 hybrid_clock.cc:648] HybridClock initialized: now 1790534404828072 us; error 41 us; skew 500 ppm
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:04.831460 19243 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:04.832602 19243 webserver.cc:460] Webserver started at http://127.6.4.3:43423/ using document root <none> and password file <none>
I20260927 18:40:04.833272 19243 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:04.833364 19243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:04.865736 19243 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.000s
I20260927 18:40:04.872893 19261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.874540 19243 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260927 18:40:04.874677 19243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "04b0a49f371e4073832232fb55850d9c"
format_stamp: "Formatted at 2026-09-27 18:40:03 on dist-test-slave-bzzj"
server_key: "0c2cb66d04fe1be8cdc71fe83a35eced"
server_key_iv: "b06e10c927459408b8b508fcaaaf48cb"
server_key_version: "kuduclusterkey@0"
I20260927 18:40:04.875422 19243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:04.891088 19243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:04.893945 19243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:04.894132 19243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:04.894742 19243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:04.895776 19243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:04.895825 19243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.895877 19243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:04.895900 19243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:04.912329 19243 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:44723
I20260927 18:40:04.912479 19374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:44723 every 8 connection(s)
I20260927 18:40:04.913580 19243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:04.919607 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19243
Sep 27 18:40:04 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534404, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:04.933909 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.920443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.6.4.62:42599)
0927 18:40:04.921296 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:04.921301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:04.923132 (+ 1831us) server_negotiation.cc:407] Connection header received
0927 18:40:04.923259 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:04.923262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:04.923302 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:04.923371 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:04.925146 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.925707 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.926607 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.926837 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.926924 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.929647 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:04.929651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:04.929653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:04.929679 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:04.931418 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.932377 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.932380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.932382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.932488 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.932807 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.932810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.932812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.933010 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:04.933205 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:04.933600 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 18:40:04.933714 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"spinlock_wait_cycles":1280,"thread_start_us":88,"threads_started":1}
I20260927 18:40:04.934831 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.920354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:34201 (local address 127.6.4.62:42599)
0927 18:40:04.920690 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:04.920695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:04.921754 (+ 1059us) server_negotiation.cc:407] Connection header received
0927 18:40:04.922836 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:04.922839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:04.922896 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:04.922976 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:04.924467 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.925234 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.926904 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.927149 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.927232 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.930235 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:04.930238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:04.930241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:04.930284 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:04.931836 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.932375 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.932378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.932380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.932499 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:04.933072 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:04.933075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:04.933077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:04.933254 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:04.933317 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:04.934621 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0927 18:40:04.934728 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"spinlock_wait_cycles":3456}
I20260927 18:40:04.936375 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:04.919540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42599 (local address 127.6.4.3:34201)
0927 18:40:04.921609 (+ 2069us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:04.921652 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:04.922575 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:04.923222 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:04.923232 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:04.923597 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:04.924297 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:04.924321 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.925370 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.925373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:04.926783 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:04.926793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:04.927235 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:04.927292 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:04.927498 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:04.928026 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:04.928060 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:04.930092 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:04.931935 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:04.931941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:04.931956 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:04.932295 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:04.932791 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:04.932794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:04.932796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:04.932963 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:04.933911 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:04.933917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:04.934248 (+ 331us) client_negotiation.cc:769] Sending connection context
0927 18:40:04.935057 (+ 809us) client_negotiation.cc:240] Negotiation successful
0927 18:40:04.935369 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1864,"thread_start_us":142,"threads_started":1}
I20260927 18:40:04.937508 19375 heartbeater.cc:347] Connected to a master server at 127.6.4.62:42599
I20260927 18:40:04.937744 19375 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.938217 19375 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
W20260927 18:40:04.939599 19375 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:42599 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:40:04.939709 19375 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.939859 19375 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:04.940640 19375 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:04.940846 19375 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
W20260927 18:40:04.941529 19375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 18:40:05 dist-test-slave-bzzj krb5kdc[17913](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534405, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:05.439934 19107 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:05.440258 19107 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:05.441412 18909 ts_manager.cc:194] Registered new tserver with Master: 57231e9cce1b4c39982d1eedf9a72a88 (127.6.4.1:41743)
I20260927 18:40:05.442920 18909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:37607
I20260927 18:40:05.696581 19241 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:05.696908 19241 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:05.697832 18909 ts_manager.cc:194] Registered new tserver with Master: 2bbe36fe654f4e06a65d66f095287af2 (127.6.4.2:36531)
I20260927 18:40:05.698652 18909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:59219
Sep 27 18:40:05 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534405, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
Sep 27 18:40:05 dist-test-slave-bzzj krb5kdc[17913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534405, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for HTTP/127.6.4.20@KRBTEST.COM
I20260927 18:40:05.941839 19375 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:05.942116 19375 heartbeater.cc:511] Master 127.6.4.62:42599 requested a full tablet report, sending...
I20260927 18:40:05.943231 18909 ts_manager.cc:194] Registered new tserver with Master: 04b0a49f371e4073832232fb55850d9c (127.6.4.3:44723)
I20260927 18:40:05.943974 18909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:34201
I20260927 18:40:05.944049 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:05.975697 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:05.949273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.6.4.62:42599)
0927 18:40:05.956306 (+ 7033us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:05.956311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:05.956598 (+ 287us) server_negotiation.cc:407] Connection header received
0927 18:40:05.956653 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:05.956657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:05.956711 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:05.956790 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:05.957769 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:05.958645 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:05.959875 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:05.960433 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:05.960529 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:05.962989 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:05.962993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:05.962996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:05.963026 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:05.967410 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:05.973703 (+ 6293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:05.973708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:05.973710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:05.973815 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:05.974229 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:05.974233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:05.974235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:05.974486 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:05.974634 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:05.975356 (+ 722us) server_negotiation.cc:299] Negotiation successful
0927 18:40:05.975494 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6928,"thread_start_us":94,"threads_started":1}
I20260927 18:40:05.980193 18909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:40:05.985392 18909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:40:06.016614 19409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.004480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:41743 (local address 127.0.0.1:38250)
0927 18:40:06.007567 (+ 3087us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:06.007591 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:06.007685 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:06.009313 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:06.009320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:06.009345 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:06.009482 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.009488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.011815 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.011819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.013274 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.013285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.015338 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.015342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.015454 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.015495 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:40:06.015609 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 18:40:06.016433 (+ 824us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3027,"mutex_wait_us":665,"thread_start_us":105,"threads_started":1}
I20260927 18:40:06.017490 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.005604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.6.4.1:41743)
0927 18:40:06.007430 (+ 1826us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:06.007438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:06.007780 (+ 342us) server_negotiation.cc:407] Connection header received
0927 18:40:06.007868 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:06.007874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:06.008948 (+ 1074us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:06.009108 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:06.009630 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.011693 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.013410 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.015032 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.015129 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.015282 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:06.017236 (+ 1954us) server_negotiation.cc:299] Negotiation successful
0927 18:40:06.017347 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1712,"thread_start_us":91,"threads_started":1}
I20260927 18:40:06.018224 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.006921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36531 (local address 127.0.0.1:42354)
0927 18:40:06.013274 (+ 6353us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:06.013290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:06.013378 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:06.014154 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:06.014157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:06.014174 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:06.014259 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.014266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.015573 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.015576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.016626 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.016636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.017841 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.017845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.017928 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.017951 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:40:06.018015 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:40:06.018093 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6282,"thread_start_us":84,"threads_started":1}
I20260927 18:40:06.021066 19041 tablet_service.cc:1467] Processing CreateTablet for tablet 5780b8e5c86d41e6939123e052afaba4 (DEFAULT_TABLE table=test-table [id=92c95bb1c1f94797856fd9a9c82ba230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:06.022264 19041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780b8e5c86d41e6939123e052afaba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:06.024499 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.004435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:44723 (local address 127.0.0.1:56318)
0927 18:40:06.005831 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:06.005888 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:06.006016 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:06.019447 (+ 13431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:06.019450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:06.019469 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:06.019554 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.019560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.021166 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.021169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.022201 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.022212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.024130 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.024133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.024219 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.024244 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 18:40:06.024294 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 18:40:06.024367 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":108,"threads_started":1}
I20260927 18:40:06.025398 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.005291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.6.4.3:44723)
0927 18:40:06.015883 (+ 10592us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:06.015891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:06.015915 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:40:06.015988 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:06.015993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:06.016229 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:06.019724 (+ 3495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:06.019885 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.021027 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.022358 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.023982 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.024967 (+ 985us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.025117 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:06.025169 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 18:40:06.025261 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10474,"thread_start_us":118,"threads_started":1}
I20260927 18:40:06.029074 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.007109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.6.4.2:36531)
0927 18:40:06.007478 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:06.007485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:06.013477 (+ 5992us) server_negotiation.cc:407] Connection header received
0927 18:40:06.013568 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:06.013574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:06.013809 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:06.013982 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:06.014382 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.015457 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.016755 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.017532 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.017632 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.017778 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:06.028803 (+ 11025us) server_negotiation.cc:299] Negotiation successful
0927 18:40:06.028936 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":83,"threads_started":1}
I20260927 18:40:06.028855 19308 tablet_service.cc:1467] Processing CreateTablet for tablet 5780b8e5c86d41e6939123e052afaba4 (DEFAULT_TABLE table=test-table [id=92c95bb1c1f94797856fd9a9c82ba230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:06.029984 19308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780b8e5c86d41e6939123e052afaba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:06.032166 19169 tablet_service.cc:1467] Processing CreateTablet for tablet 5780b8e5c86d41e6939123e052afaba4 (DEFAULT_TABLE table=test-table [id=92c95bb1c1f94797856fd9a9c82ba230]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:06.033221 19169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780b8e5c86d41e6939123e052afaba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:06.040809 19411 tablet_bootstrap.cc:492] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Bootstrap starting.
I20260927 18:40:06.043258 19411 tablet_bootstrap.cc:654] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:06.044318 19411 log.cc:824] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:06.046715 19411 tablet_bootstrap.cc:492] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: No bootstrap required, opened a new log
I20260927 18:40:06.046933 19411 ts_tablet_manager.cc:1397] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:40:06.051239 19413 tablet_bootstrap.cc:492] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Bootstrap starting.
I20260927 18:40:06.051800 19412 tablet_bootstrap.cc:492] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: Bootstrap starting.
I20260927 18:40:06.051681 19411 raft_consensus.cc:348] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.051908 19411 raft_consensus.cc:374] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:06.051962 19411 raft_consensus.cc:729] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b0a49f371e4073832232fb55850d9c, State: Initialized, Role: FOLLOWER
I20260927 18:40:06.052465 19411 consensus_queue.cc:260] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.053131 19375 heartbeater.cc:503] Master 127.6.4.62:42599 was elected leader, sending a full tablet report...
I20260927 18:40:06.053442 19411 ts_tablet_manager.cc:1428] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 18:40:06.061928 19413 tablet_bootstrap.cc:654] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:06.063488 19412 tablet_bootstrap.cc:654] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:06.064491 19413 log.cc:824] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:06.065423 19412 log.cc:824] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:06.068739 19412 tablet_bootstrap.cc:492] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: No bootstrap required, opened a new log
I20260927 18:40:06.069020 19412 ts_tablet_manager.cc:1397] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260927 18:40:06.074749 19412 raft_consensus.cc:348] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.076893 19413 tablet_bootstrap.cc:492] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: No bootstrap required, opened a new log
I20260927 18:40:06.077127 19413 ts_tablet_manager.cc:1397] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.000s
I20260927 18:40:06.082057 19413 raft_consensus.cc:348] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.082507 19412 raft_consensus.cc:374] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:06.082649 19412 raft_consensus.cc:729] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57231e9cce1b4c39982d1eedf9a72a88, State: Initialized, Role: FOLLOWER
I20260927 18:40:06.083352 19412 consensus_queue.cc:260] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.083629 19413 raft_consensus.cc:374] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:06.083747 19413 raft_consensus.cc:729] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbe36fe654f4e06a65d66f095287af2, State: Initialized, Role: FOLLOWER
I20260927 18:40:06.084352 19413 consensus_queue.cc:260] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.085636 19412 ts_tablet_manager.cc:1428] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260927 18:40:06.086084 19107 heartbeater.cc:503] Master 127.6.4.62:42599 was elected leader, sending a full tablet report...
I20260927 18:40:06.088196 19413 ts_tablet_manager.cc:1428] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260927 18:40:06.088637 19241 heartbeater.cc:503] Master 127.6.4.62:42599 was elected leader, sending a full tablet report...
I20260927 18:40:06.090134 19419 raft_consensus.cc:482] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:06.090364 19419 raft_consensus.cc:504] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.094847 19419 leader_election.cc:288] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04b0a49f371e4073832232fb55850d9c (127.6.4.3:44723), 2bbe36fe654f4e06a65d66f095287af2 (127.6.4.2:36531)
I20260927 18:40:06.099938 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.092267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44859 (local address 127.6.4.3:44723)
0927 18:40:06.092387 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:06.092392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:06.095247 (+ 2855us) server_negotiation.cc:407] Connection header received
0927 18:40:06.095455 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:06.095459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:06.095513 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:06.095558 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:06.096006 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.096852 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.098563 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.099193 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.099294 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.099363 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:06.099712 (+ 349us) server_negotiation.cc:299] Negotiation successful
0927 18:40:06.099804 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:40:06.101240 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.091849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:44723 (local address 127.6.4.1:44859)
0927 18:40:06.095152 (+ 3303us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:06.095167 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:06.095329 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:06.095719 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:06.095722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:06.095740 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:06.095880 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.095886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.097006 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.097010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.098402 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.098413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.099438 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.099442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.099559 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.099585 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 18:40:06.100943 (+ 1358us) client_negotiation.cc:240] Negotiation successful
0927 18:40:06.101030 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3236,"thread_start_us":121,"threads_started":1}
I20260927 18:40:06.102692 19318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780b8e5c86d41e6939123e052afaba4" candidate_uuid: "57231e9cce1b4c39982d1eedf9a72a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b0a49f371e4073832232fb55850d9c" is_pre_election: true
I20260927 18:40:06.103080 19318 raft_consensus.cc:2454] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57231e9cce1b4c39982d1eedf9a72a88 in term 0.
I20260927 18:40:06.103726 18994 leader_election.cc:302] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b0a49f371e4073832232fb55850d9c, 57231e9cce1b4c39982d1eedf9a72a88; no voters:
I20260927 18:40:06.104008 19419 raft_consensus.cc:2781] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:06.104130 19419 raft_consensus.cc:482] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:06.104180 19419 raft_consensus.cc:3037] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:06.105408 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.094783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:39255 (local address 127.6.4.2:36531)
0927 18:40:06.094892 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:06.094897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:06.096881 (+ 1984us) server_negotiation.cc:407] Connection header received
0927 18:40:06.097064 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:06.097067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:06.097112 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:06.097161 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:06.097558 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.098132 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.104108 (+ 5976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.104706 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.104789 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.104848 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:06.105207 (+ 359us) server_negotiation.cc:299] Negotiation successful
0927 18:40:06.105292 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:40:06.105561 19419 raft_consensus.cc:504] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.105996 19419 leader_election.cc:288] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [CANDIDATE]: Term 1 election: Requested vote from peers 04b0a49f371e4073832232fb55850d9c (127.6.4.3:44723), 2bbe36fe654f4e06a65d66f095287af2 (127.6.4.2:36531)
I20260927 18:40:06.106657 19318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780b8e5c86d41e6939123e052afaba4" candidate_uuid: "57231e9cce1b4c39982d1eedf9a72a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b0a49f371e4073832232fb55850d9c"
I20260927 18:40:06.106788 19318 raft_consensus.cc:3037] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:06.107846 19318 raft_consensus.cc:2454] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57231e9cce1b4c39982d1eedf9a72a88 in term 1.
I20260927 18:40:06.108052 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.094437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36531 (local address 127.6.4.1:39255)
0927 18:40:06.096803 (+ 2366us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:06.096816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:06.096941 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:06.097296 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:06.097299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:06.097315 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:06.097444 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.097450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.098238 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.098241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.099278 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:06.099287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.104945 (+ 5658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.104949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:06.105061 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.105084 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:40:06.105590 (+ 506us) client_negotiation.cc:240] Negotiation successful
0927 18:40:06.105671 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2303,"thread_start_us":101,"threads_started":1}
I20260927 18:40:06.108192 18994 leader_election.cc:302] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b0a49f371e4073832232fb55850d9c, 57231e9cce1b4c39982d1eedf9a72a88; no voters:
I20260927 18:40:06.108428 19419 raft_consensus.cc:2781] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:06.108515 19419 raft_consensus.cc:686] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [term 1 LEADER]: Becoming Leader. State: Replica: 57231e9cce1b4c39982d1eedf9a72a88, State: Running, Role: LEADER
I20260927 18:40:06.108896 19419 consensus_queue.cc:237] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } }
I20260927 18:40:06.108975 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780b8e5c86d41e6939123e052afaba4" candidate_uuid: "57231e9cce1b4c39982d1eedf9a72a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bbe36fe654f4e06a65d66f095287af2" is_pre_election: true
I20260927 18:40:06.109279 19190 raft_consensus.cc:2454] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57231e9cce1b4c39982d1eedf9a72a88 in term 0.
I20260927 18:40:06.109606 19195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780b8e5c86d41e6939123e052afaba4" candidate_uuid: "57231e9cce1b4c39982d1eedf9a72a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bbe36fe654f4e06a65d66f095287af2"
I20260927 18:40:06.109707 19195 raft_consensus.cc:3037] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:06.110911 19195 raft_consensus.cc:2454] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57231e9cce1b4c39982d1eedf9a72a88 in term 1.
I20260927 18:40:06.113730 18908 catalog_manager.cc:5495] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57231e9cce1b4c39982d1eedf9a72a88 (127.6.4.1). New cstate: current_term: 1 leader_uuid: "57231e9cce1b4c39982d1eedf9a72a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57231e9cce1b4c39982d1eedf9a72a88" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 } health_report { overall_health: UNKNOWN } } }
I20260927 18:40:06.147202 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.141881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38266 (local address 127.6.4.1:41743)
0927 18:40:06.142009 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:06.142013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:06.142028 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:40:06.142124 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:06.142127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:06.142198 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:06.142310 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:06.142733 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.143543 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.144868 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:06.145198 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:06.145301 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:06.145765 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:06.146379 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:06.146512 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:06.146964 (+ 452us) server_negotiation.cc:299] Negotiation successful
0927 18:40:06.147031 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 18:40:06.160334 19108 tablet.cc:2369] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:06.162858 19318 raft_consensus.cc:1261] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 1 FOLLOWER]: Refusing update from remote peer 57231e9cce1b4c39982d1eedf9a72a88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:06.163815 19419 consensus_queue.cc:1035] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04b0a49f371e4073832232fb55850d9c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:40:06.165328 19376 tablet.cc:2369] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:06.166376 19195 raft_consensus.cc:1261] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 1 FOLLOWER]: Refusing update from remote peer 57231e9cce1b4c39982d1eedf9a72a88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:06.167094 19419 consensus_queue.cc:1035] T 5780b8e5c86d41e6939123e052afaba4 P 57231e9cce1b4c39982d1eedf9a72a88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bbe36fe654f4e06a65d66f095287af2" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:40:06.172765 19242 tablet.cc:2369] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:06.182734 19426 mvcc.cc:205] Tried to move back new op lower bound from 7334028927615655936 to 7334028927427772416. Current Snapshot: MvccSnapshot[applied={T|T < 7334028927615655936}]
I20260927 18:40:06.187336 18908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:06.187664 18908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:06.188385 19425 mvcc.cc:205] Tried to move back new op lower bound from 7334028927615655936 to 7334028927427772416. Current Snapshot: MvccSnapshot[applied={T|T < 7334028927615655936 or (T in {7334028927615655936})}]
I20260927 18:40:06.190600 18908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fda5b30931047bcb83f876e355546e1: Sending DeleteTablet for 3 replicas of tablet 5780b8e5c86d41e6939123e052afaba4
I20260927 18:40:06.191430 19308 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780b8e5c86d41e6939123e052afaba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:06 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:56318
I20260927 18:40:06.191846 19428 mvcc.cc:205] Tried to move back new op lower bound from 7334028927615655936 to 7334028927427772416. Current Snapshot: MvccSnapshot[applied={T|T < 7334028927615655936}]
I20260927 18:40:06.192588 19041 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780b8e5c86d41e6939123e052afaba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:06 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:38250
I20260927 18:40:06.193470 19169 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780b8e5c86d41e6939123e052afaba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:06 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:42354
I20260927 18:40:06.194907 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 18947
I20260927 18:40:06.195814 19434 tablet_replica.cc:330] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: stopping tablet replica
I20260927 18:40:06.196201 19434 raft_consensus.cc:2229] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:06.196494 19434 raft_consensus.cc:2258] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:06.196882 19436 tablet_replica.cc:330] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: stopping tablet replica
I20260927 18:40:06.197187 19436 raft_consensus.cc:2229] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:06.197479 19436 raft_consensus.cc:2258] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:06.199157 19436 ts_tablet_manager.cc:1905] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:06.201960 19436 ts_tablet_manager.cc:1918] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:06.202067 19436 log.cc:1196] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/5780b8e5c86d41e6939123e052afaba4
I20260927 18:40:06.203819 19434 ts_tablet_manager.cc:1905] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:06.206758 19434 ts_tablet_manager.cc:1918] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:06.206951 19434 log.cc:1196] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/5780b8e5c86d41e6939123e052afaba4
I20260927 18:40:06.207367 19434 ts_tablet_manager.cc:1939] T 5780b8e5c86d41e6939123e052afaba4 P 04b0a49f371e4073832232fb55850d9c: Deleting consensus metadata
I20260927 18:40:06.207966 19436 ts_tablet_manager.cc:1939] T 5780b8e5c86d41e6939123e052afaba4 P 2bbe36fe654f4e06a65d66f095287af2: Deleting consensus metadata
I20260927 18:40:06.208958 18893 catalog_manager.cc:4841] TS 04b0a49f371e4073832232fb55850d9c (127.6.4.3:44723): tablet 5780b8e5c86d41e6939123e052afaba4 (table test-table [id=92c95bb1c1f94797856fd9a9c82ba230]) successfully deleted
I20260927 18:40:06.209242 18894 catalog_manager.cc:4841] TS 2bbe36fe654f4e06a65d66f095287af2 (127.6.4.2:36531): tablet 5780b8e5c86d41e6939123e052afaba4 (table test-table [id=92c95bb1c1f94797856fd9a9c82ba230]) successfully deleted
W20260927 18:40:06.209810 18895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:41743 (error 108)
I20260927 18:40:06.211252 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19111
I20260927 18:40:06.211978 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:06.210991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:41743 (local address 127.0.0.1:38270)
0927 18:40:06.211808 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:06.211888 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:41743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720}
W20260927 18:40:06.215816 18895 catalog_manager.cc:4584] TS 57231e9cce1b4c39982d1eedf9a72a88 (127.6.4.1:41743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5780b8e5c86d41e6939123e052afaba4: Network error: Client connection negotiation failed: client connection to 127.6.4.1:41743: connect: Connection refused (error 111)
I20260927 18:40:06.220238 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19243
I20260927 18:40:06.226763 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 18876
I20260927 18:40:06.234366 6160 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 18:40:06.568243 6160 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 18:40:06.568351 6160 mini_ranger.cc:66] Stopping Ranger...
I20260927 18:40:06.903105 6160 mini_ranger.cc:68] Stopped Ranger
2026-09-27 18:40:06.903 UTC [17943] LOG: received smart shutdown request
2026-09-27 18:40:06.906 UTC [17943] LOG: background worker "logical replication launcher" (PID 17952) exited with exit code 1
2026-09-27 18:40:06.907 UTC [17947] LOG: shutting down
2026-09-27 18:40:07.046 UTC [17943] LOG: database system is shut down
2026-09-27T18:40:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46344 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:40:07 dist-test-slave-bzzj krb5kdc[19446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:40:07 dist-test-slave-bzzj krb5kdc[19446](info): set up 2 sockets
Sep 27 18:40:07 dist-test-slave-bzzj krb5kdc[19446](info): commencing operation
krb5kdc: starting...
W20260927 18:40:08.091838 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:40:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:40:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:40:08.204946 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45955
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:45955,127.6.4.61:38921,127.6.4.60:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:08.311864 19459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:08.312144 19459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:08.312186 19459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:08.316510 19459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:08.316567 19459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:08.316586 19459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:08.316601 19459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:08.316613 19459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:08.321805 19459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:45955,127.6.4.61:38921,127.6.4.60:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:08.322775 19459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:08.323767 19459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:08.331885 19464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:08.331898 19467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:08.331885 19465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:08.332244 19459 server_base.cc:1034] running on GCE node
I20260927 18:40:08.332798 19459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:08.333984 19459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:08.335219 19459 hybrid_clock.cc:648] HybridClock initialized: now 1790534408335190 us; error 44 us; skew 500 ppm
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:08.337766 19459 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:08.338626 19459 webserver.cc:460] Webserver started at http://127.6.4.62:45147/ using document root <none> and password file <none>
I20260927 18:40:08.339144 19459 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:08.339208 19459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:08.339390 19459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:08.341444 19459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "1192ec60fcce4d95901c7590d6df45d8"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "aeb7ba26ffdaabb5ef109857649f3137"
server_key_iv: "9dcb6c2ffcc2a7fe0f684dae350881df"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.341977 19459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "1192ec60fcce4d95901c7590d6df45d8"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "aeb7ba26ffdaabb5ef109857649f3137"
server_key_iv: "9dcb6c2ffcc2a7fe0f684dae350881df"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.345504 19459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:08.347762 19474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:08.348922 19459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:08.349038 19459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "1192ec60fcce4d95901c7590d6df45d8"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "aeb7ba26ffdaabb5ef109857649f3137"
server_key_iv: "9dcb6c2ffcc2a7fe0f684dae350881df"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.349125 19459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:08.373720 19459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:08.377594 19459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:08.377796 19459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:08.384568 19459 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:45955
I20260927 18:40:08.384585 19526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:45955 every 8 connection(s)
I20260927 18:40:08.385515 19459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:08.388268 19527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:08.390803 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19459
I20260927 18:40:08.390908 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:40:08.391202 6160 external_mini_cluster.cc:1428] Setting key 849d900cd5f0819fc53ab27d4eb51b1d
I20260927 18:40:08.392122 19527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.4.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.61@KRBTEST.COM" created.
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:08.406345 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.394464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45955 (local address 127.0.0.1:37138)
0927 18:40:08.395020 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.395064 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.395904 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.396565 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.396570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.396985 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.397219 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.397232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.398315 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.398318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.399064 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.399074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.399627 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.399676 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.399768 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.400317 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.400333 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.401845 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.403471 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.403480 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.403489 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.403692 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.404065 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.404067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.404068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.404209 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.404953 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.404958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.405078 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.405213 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.405480 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":140,"threads_started":1}
I20260927 18:40:08.406358 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.395439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.6.4.62:45955)
0927 18:40:08.395719 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.395729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.395771 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:40:08.396205 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.396209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.396384 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.396557 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.397349 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.398199 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.399171 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.399541 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.399677 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.401975 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.401981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.401990 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.402009 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.403372 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.403816 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.403823 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.403825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.403971 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.404292 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.404295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.404297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.404807 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.404947 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.405314 (+ 367us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.405599 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260927 18:40:08.408939 19527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:40:08.409574 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.409353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58156)
0927 18:40:08.409451 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.409516 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:40:08.409783 19476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
I20260927 18:40:08.410495 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.410316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58166)
0927 18:40:08.410402 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.410446 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:40:08.410612 19527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:38921: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab.
Entry for principal kudu/127.6.4.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab.
I20260927 18:40:08.415162 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:38921
--webserver_interface=127.6.4.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab
--principal=kudu/127.6.4.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:45955,127.6.4.61:38921,127.6.4.60:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:08.459898 19527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } attempt: 1
I20260927 18:40:08.460768 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.460472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58170)
0927 18:40:08.460622 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.460687 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 18:40:08.461506 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.461283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58178)
0927 18:40:08.461399 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.461446 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260927 18:40:08.461618 19527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:38921: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
W20260927 18:40:08.522675 19534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:08.522941 19534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:08.522998 19534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:08.526954 19534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:08.527007 19534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:08.527034 19534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:08.527063 19534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:08.527088 19534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:08.529769 19527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } attempt: 2
I20260927 18:40:08.530519 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.530241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58190)
0927 18:40:08.530376 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.530444 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 18:40:08.531229 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.531026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58194)
0927 18:40:08.531131 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.531174 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:40:08.531335 19527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:38921: Network error: Client connection negotiation failed: client connection to 127.6.4.61:38921: connect: Connection refused (error 111)
I20260927 18:40:08.532498 19534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:45955,127.6.4.61:38921,127.6.4.60:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab
--principal=kudu/127.6.4.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:38921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.4.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:08.533510 19534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:08.534564 19534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:08.542512 19542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:08.542548 19539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:08.542531 19540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:08.542861 19534 server_base.cc:1034] running on GCE node
I20260927 18:40:08.543366 19534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:08.544440 19534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:08.545588 19534 hybrid_clock.cc:648] HybridClock initialized: now 1790534408545560 us; error 40 us; skew 500 ppm
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:08.548162 19534 init.cc:377] Logged in from keytab as kudu/127.6.4.61@KRBTEST.COM (short username kudu)
I20260927 18:40:08.548985 19534 webserver.cc:460] Webserver started at http://127.6.4.61:46651/ using document root <none> and password file <none>
I20260927 18:40:08.549490 19534 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:08.549544 19534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:08.549708 19534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:08.551648 19534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data/instance:
uuid: "cd570cf9319a4c1090ee7c2fb3048470"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "55afc46ff70f9ba2db24941bc1f3bf5a"
server_key_iv: "bf6800778a5ca6a43e42307855d00bf5"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.552208 19534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal/instance:
uuid: "cd570cf9319a4c1090ee7c2fb3048470"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "55afc46ff70f9ba2db24941bc1f3bf5a"
server_key_iv: "bf6800778a5ca6a43e42307855d00bf5"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.555732 19534 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:08.558087 19549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:08.559259 19534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:08.559355 19534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal
uuid: "cd570cf9319a4c1090ee7c2fb3048470"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "55afc46ff70f9ba2db24941bc1f3bf5a"
server_key_iv: "bf6800778a5ca6a43e42307855d00bf5"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.559437 19534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:08.583461 19534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:08.586341 19534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:08.586491 19534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:08.593076 19534 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.61:38921
I20260927 18:40:08.593186 19601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.61:38921 every 8 connection(s)
I20260927 18:40:08.594033 19534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
I20260927 18:40:08.596783 19602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:08.600529 19602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:40:08.601187 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19534
I20260927 18:40:08.601290 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-1/wal/instance
I20260927 18:40:08.601553 6160 external_mini_cluster.cc:1428] Setting key 7f85ee45dd25b188f10ebe31ebd99570
WARNING: no policy specified for kudu/127.6.4.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.60@KRBTEST.COM" created.
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:08.615926 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.602468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.6.4.62:45955)
0927 18:40:08.602596 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.602601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.603608 (+ 1007us) server_negotiation.cc:407] Connection header received
0927 18:40:08.604597 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.604602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.604661 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.604755 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.605883 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.606746 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.607856 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.608246 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.608356 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.612482 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.612486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.612489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.612521 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.613900 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.614434 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.614436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.614437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.614501 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.614809 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.614811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.614812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.614980 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.615050 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.615699 (+ 649us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.615803 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:40:08.617153 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.602932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45955 (local address 127.0.0.1:37146)
0927 18:40:08.603437 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.603499 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.604361 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.604901 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.604907 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.605235 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.605732 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.605745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.606869 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.606872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.607738 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.607746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.608344 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.608391 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.608541 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.609093 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.609115 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.612289 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.614016 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.614022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.614038 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.614353 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.614568 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.614571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.614573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.614733 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.615073 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.615080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.615431 (+ 351us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.615643 (+ 212us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.616009 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":143,"threads_started":1}
I20260927 18:40:08.618571 19602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:40:08.622484 19527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } attempt: 3
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab.
Entry for principal kudu/127.6.4.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab.
I20260927 18:40:08.627223 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.619065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58208)
0927 18:40:08.619218 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.619229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.619286 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.619781 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.619784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.619946 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.620038 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.620043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.621041 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.621045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.621571 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.621579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.622128 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.622143 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.622228 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.622692 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.622704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.623952 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.625760 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.625763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.625765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.625927 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.626193 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.626196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.626198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.626281 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.626742 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.626745 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.626831 (+ 86us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.626933 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.627049 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260927 18:40:08.627487 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:42221
--webserver_interface=127.6.4.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab
--principal=kudu/127.6.4.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:41761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:45955,127.6.4.61:38921,127.6.4.60:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:08.627712 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.619165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.6.4.61:38921)
0927 18:40:08.619455 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.619464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.619481 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:40:08.619513 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.619517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.619674 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.619788 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.620160 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.620931 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.621696 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.622044 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.622140 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.624081 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.624119 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.624133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.624156 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.625673 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.626018 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.626025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.626026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.626118 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.626369 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.626371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.626373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.626645 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.626769 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.627079 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.627227 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":112,"threads_started":1}
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:40:08.629753 19602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:40:08.630420 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.630162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51160)
0927 18:40:08.630276 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.630352 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:40:08.630614 19551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
I20260927 18:40:08.631424 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.631242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51174)
0927 18:40:08.631321 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.631363 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260927 18:40:08.631569 19602 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:42221: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
I20260927 18:40:08.632378 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.622913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58212 (local address 127.6.4.61:38921)
0927 18:40:08.623169 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.623176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.623186 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:40:08.623212 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.623213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.623245 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.623317 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.624148 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.624823 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.625597 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.625868 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.625952 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.628396 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.628399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.628402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.628423 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.630612 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.631067 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.631070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.631072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.631155 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.631421 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.631424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.631425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.631791 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.631863 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.632169 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.632279 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260927 18:40:08.632426 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.622832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58212)
0927 18:40:08.622951 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.622965 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.623044 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.623369 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.623372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.623512 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.623995 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.624004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.624925 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.624929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.625492 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.625501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.625952 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.625968 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.626070 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.626847 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.626860 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.628264 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.630705 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.630708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.630711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.630948 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.631235 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.631238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.631239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.631330 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.631890 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.631893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.631991 (+ 98us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.632121 (+ 130us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.632261 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:40:08.633368 19527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:40:08.634002 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.633739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51180)
0927 18:40:08.633857 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.633929 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 18:40:08.634696 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.634479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51194)
0927 18:40:08.634601 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.634641 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260927 18:40:08.634805 19527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:42221: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
I20260927 18:40:08.665948 19527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } attempt: 1
I20260927 18:40:08.666846 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.666520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51202)
0927 18:40:08.666690 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.666763 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260927 18:40:08.667544 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.667339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51214)
0927 18:40:08.667445 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.667491 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 18:40:08.667656 19527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:42221: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
I20260927 18:40:08.680776 19602 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } attempt: 1
I20260927 18:40:08.681614 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.681277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51222)
0927 18:40:08.681443 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.681522 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 18:40:08.682307 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.682087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51224)
0927 18:40:08.682185 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.682236 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 18:40:08.682454 19602 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:42221: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
W20260927 18:40:08.740509 19609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:08.740752 19609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:08.740798 19609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 18:40:08.742822 19527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } attempt: 2
I20260927 18:40:08.743641 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.743341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51232)
0927 18:40:08.743493 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.743557 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 18:40:08.744347 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.744141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51240)
0927 18:40:08.744242 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.744290 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:40:08.744498 19527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:42221: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
W20260927 18:40:08.745196 19609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:08.745247 19609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:08.745275 19609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:08.745304 19609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:08.745332 19609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:08.750610 19602 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } attempt: 2
I20260927 18:40:08.750793 19609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:41761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:45955,127.6.4.61:38921,127.6.4.60:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab
--principal=kudu/127.6.4.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.4.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:08.751349 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.751060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51244)
0927 18:40:08.751204 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.751270 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:40:08.752019 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.751801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51248)
0927 18:40:08.751896 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.751944 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 18:40:08.752173 19609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260927 18:40:08.752189 19602 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:42221: Network error: Client connection negotiation failed: client connection to 127.6.4.60:42221: connect: Connection refused (error 111)
I20260927 18:40:08.753530 19609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:08.763471 19618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:08.763484 19616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:08.763507 19615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:08.763746 19609 server_base.cc:1034] running on GCE node
I20260927 18:40:08.764413 19609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:08.765520 19609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:08.766693 19609 hybrid_clock.cc:648] HybridClock initialized: now 1790534408766673 us; error 34 us; skew 500 ppm
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:08.769338 19609 init.cc:377] Logged in from keytab as kudu/127.6.4.60@KRBTEST.COM (short username kudu)
I20260927 18:40:08.770195 19609 webserver.cc:460] Webserver started at http://127.6.4.60:35643/ using document root <none> and password file <none>
I20260927 18:40:08.770727 19609 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:08.770792 19609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:08.770972 19609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:08.773036 19609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data/instance:
uuid: "f14e07b9e57041aaa5d76d07cce4e428"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "b8c68cbf28b92061efa714bb1b5a72c3"
server_key_iv: "2c6e0197e8aa68af0142b5073668c6ea"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.773558 19609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal/instance:
uuid: "f14e07b9e57041aaa5d76d07cce4e428"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "b8c68cbf28b92061efa714bb1b5a72c3"
server_key_iv: "2c6e0197e8aa68af0142b5073668c6ea"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.777113 19609 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:08.779441 19625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:08.780652 19609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:08.780761 19609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal
uuid: "f14e07b9e57041aaa5d76d07cce4e428"
format_stamp: "Formatted at 2026-09-27 18:40:08 on dist-test-slave-bzzj"
server_key: "b8c68cbf28b92061efa714bb1b5a72c3"
server_key_iv: "2c6e0197e8aa68af0142b5073668c6ea"
server_key_version: "encryptionkey@0"
I20260927 18:40:08.780850 19609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:08.807543 19609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:08.810763 19609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:08.810945 19609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:08.817799 19609 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.60:42221
I20260927 18:40:08.817824 19677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.60:42221 every 8 connection(s)
I20260927 18:40:08.818765 19609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
I20260927 18:40:08.821591 19678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:08.825100 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19609
I20260927 18:40:08.825196 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534257539224-6160-0/minicluster-data/master-2/wal/instance
I20260927 18:40:08.825465 6160 external_mini_cluster.cc:1428] Setting key 92eca69502930a4bc58d3e91317058e9
I20260927 18:40:08.825444 19678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:40:08.836920 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.827270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.6.4.62:45955)
0927 18:40:08.827386 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.827391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.827663 (+ 272us) server_negotiation.cc:407] Connection header received
0927 18:40:08.828276 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.828279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.828335 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.828403 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.829213 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.829709 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.830658 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.830869 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.830955 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.833329 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.833332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.833334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.833359 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.834834 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.835395 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.835398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.835400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.835483 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.835814 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.835817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.835818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.836012 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.836121 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.836628 (+ 507us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.836750 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:40:08.837749 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.827194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45955 (local address 127.0.0.1:37152)
0927 18:40:08.827531 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.827576 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.828079 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.828464 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.828469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.828814 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.829078 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.829091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.829805 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.829808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.830539 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.830548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.830951 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.831002 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.831195 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.831709 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.831727 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.833206 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.834931 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.834939 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.834947 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.835304 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.835563 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.835566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.835568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.835734 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.836131 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.836137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.836380 (+ 243us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.836540 (+ 160us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.836868 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
I20260927 18:40:08.837764 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.826814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.6.4.60:42221)
0927 18:40:08.827218 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.827237 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.827277 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:40:08.828074 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.828113 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.828312 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.828468 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.829236 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.830048 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.830840 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.831329 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.831430 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.833687 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.833692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.833704 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.833729 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.835318 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.835737 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.835742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.835743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.835820 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.836061 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.836064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.836066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.836371 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.836511 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.836754 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.836919 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
I20260927 18:40:08.838871 19678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:40:08.843354 19602 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } attempt: 3
I20260927 18:40:08.843633 19527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } attempt: 3
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:08.847030 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.839437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.6.4.61:38921)
0927 18:40:08.839561 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.839566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.839581 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:40:08.839622 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.839626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.839676 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.839753 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.840180 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.840701 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.841437 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.841681 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.841768 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.843628 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.843632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.843634 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.843662 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.845404 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.845849 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.845853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.845855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.845938 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.846219 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.846223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.846225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.846392 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.846473 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.846766 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.846871 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:40:08.847481 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.839324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:38921 (local address 127.0.0.1:58222)
0927 18:40:08.839457 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.839469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.839525 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.839783 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.839786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.839924 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.840039 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.840047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.840813 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.840816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.841333 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.841340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.841763 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.841777 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.841865 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.842248 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.842260 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.843505 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.845509 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.845512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.845514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.845736 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.846064 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.846067 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.846069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.846146 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.846478 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.846481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.846579 (+ 98us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.847206 (+ 627us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.847343 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 18:40:08.848171 19678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:40:08.849454 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.840132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.6.4.62:45955)
0927 18:40:08.840230 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.840233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.840247 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:40:08.840359 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.840363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.840398 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.840457 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.840988 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.841448 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.842163 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.842377 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.842455 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.845318 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.845322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.845325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.845351 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.847200 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.847821 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.847824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.847826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.847907 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.848228 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.848231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.848232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.848441 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.848637 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.849239 (+ 602us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.849330 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:40:08.853762 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.843691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51268)
0927 18:40:08.843800 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.843815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.843885 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.844386 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.844389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.844707 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.844804 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.844812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.845705 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.845709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.846506 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.846517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.847130 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.847148 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.847222 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.847834 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.847848 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.849330 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.852214 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.852218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.852220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.852446 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.852771 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.852774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.852776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.852855 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.853284 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.853287 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.853380 (+ 93us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.853459 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.853585 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:40:08.854270 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.843951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51280)
0927 18:40:08.844089 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.844124 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.844193 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.845029 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.845032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.845200 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.845289 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.845295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.847656 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.847659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.848432 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.848444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.849046 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.849061 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.849140 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.849549 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.849565 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.851083 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.852827 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.852830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.852832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.853061 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.853352 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.853355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.853357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.853434 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.853825 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.853828 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.853931 (+ 103us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.854006 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.854126 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:40:08.854820 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.843854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.6.4.60:42221)
0927 18:40:08.844182 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.844185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.844197 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:40:08.844228 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.844231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.844267 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.844527 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.844922 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.845590 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.846641 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.847022 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.847400 (+ 378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.849463 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.849467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.849469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.849492 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.852079 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.852553 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.852556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.852558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.852680 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.852949 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.852952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.852954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.853158 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.854253 (+ 1095us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.854520 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.854641 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":57}
I20260927 18:40:08.854893 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.844168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.6.4.60:42221)
0927 18:40:08.844543 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.844552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.844565 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:40:08.844596 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.844599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.844910 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.845387 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.845462 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.847310 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.848584 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.848931 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.850841 (+ 1910us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.851192 (+ 351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.851196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.851198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.851222 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.852727 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.853176 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.853179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.853180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.853261 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.853529 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.853532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.853534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.853732 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.854438 (+ 706us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.854525 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.854705 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":305,"spinlock_wait_cycles":2944,"thread_start_us":114,"threads_started":1}
Sep 27 18:40:08 dist-test-slave-bzzj krb5kdc[19446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:40:08.858402 19602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: Bootstrap starting.
I20260927 18:40:08.858497 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.848567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:42221 (local address 127.0.0.1:51286)
0927 18:40:08.849057 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:08.849070 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:08.849134 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:08.850255 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:08.850257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:08.850400 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:08.850456 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.850460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.851409 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.851412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.852625 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:08.852632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.853128 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.853141 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:08.853207 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.853630 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:08.853644 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:08.855537 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:08.857296 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.857299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.857301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.857463 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.857684 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:08.857687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:08.857689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:08.857758 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:08.858054 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:08.858057 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:08.858131 (+ 74us) client_negotiation.cc:769] Sending connection context
0927 18:40:08.858210 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:40:08.858330 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":19,"thread_start_us":82,"threads_started":1}
I20260927 18:40:08.858534 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.848847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.6.4.60:42221)
0927 18:40:08.850071 (+ 1224us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.850081 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.850093 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:40:08.850122 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.850124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.850160 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.850229 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.850578 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.851307 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.852727 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.853026 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.854884 (+ 1858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.855794 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.855797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.855799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.855822 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.857201 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.857539 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.857542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.857544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.857613 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.857827 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.857830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.857832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.857976 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.858053 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.858258 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.858381 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1779,"server-negotiator.queue_time_us":1061,"thread_start_us":84,"threads_started":1}
I20260927 18:40:08.858542 19527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: Bootstrap starting.
I20260927 18:40:08.860754 19678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: Bootstrap starting.
I20260927 18:40:08.861284 19602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:08.861284 19527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:08.862150 19527 log.cc:824] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:08.862150 19602 log.cc:824] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:08.862746 19678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:08.863545 19678 log.cc:824] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:08.864723 19527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: No bootstrap required, opened a new log
I20260927 18:40:08.865440 19602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: No bootstrap required, opened a new log
I20260927 18:40:08.866438 19678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: No bootstrap required, opened a new log
I20260927 18:40:08.866816 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.850655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58224 (local address 127.6.4.61:38921)
0927 18:40:08.850747 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.850801 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.850816 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:40:08.850847 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.850849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.850888 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.850956 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.851818 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.852629 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.853733 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.854040 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.854133 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.863108 (+ 8975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.863111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.863114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.863135 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.864495 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.864968 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.864971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.864973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.865067 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.865352 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.865355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.865357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.865690 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.865850 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.866549 (+ 699us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.866676 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:40:08.868736 19527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.868997 19527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:08.869060 19527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1192ec60fcce4d95901c7590d6df45d8, State: Initialized, Role: FOLLOWER
I20260927 18:40:08.869212 19602 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.869390 19602 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:08.869431 19602 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd570cf9319a4c1090ee7c2fb3048470, State: Initialized, Role: FOLLOWER
I20260927 18:40:08.869566 19527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.869925 19602 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.870316 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:40:08.870890 19527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:08.871093 19602 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:08.871289 19678 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.871492 19678 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:08.871541 19678 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14e07b9e57041aaa5d76d07cce4e428, State: Initialized, Role: FOLLOWER
I20260927 18:40:08.871986 19678 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.873147 19678 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:08.875300 19705 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.875710 19705 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:40:08.876693 19711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:40:08.876811 19711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 18:40:08.879302 19480 tablet.cc:2369] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:40:08.880913 19729 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:40:08.880983 19729 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:40:08.882468 19609 master_runner.cc:263] No leader master found from master f14e07b9e57041aaa5d76d07cce4e428
I20260927 18:40:08.882529 19609 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:40:08.883842 19459 master_runner.cc:263] No leader master found from master 1192ec60fcce4d95901c7590d6df45d8
I20260927 18:40:08.883906 19459 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:40:08.886252 19534 master_runner.cc:263] No leader master found from master cd570cf9319a4c1090ee7c2fb3048470
I20260927 18:40:08.886315 19534 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:40:08.886601 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.876275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.6.4.61:38921)
0927 18:40:08.876378 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.876382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.877322 (+ 940us) server_negotiation.cc:407] Connection header received
0927 18:40:08.877506 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.877509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.877570 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.877622 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.878583 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.879292 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.880515 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.880836 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.880909 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.882760 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.882764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.882767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.882791 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.884743 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.885203 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.885222 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.885224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.885310 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.885674 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.885677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.885679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.885895 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.885994 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.886331 (+ 337us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.886441 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 18:40:08.891456 19717 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:40:08.891547 19717 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:40:08.893153 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.876902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.6.4.60:42221)
0927 18:40:08.877248 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.877252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.884897 (+ 7645us) server_negotiation.cc:407] Connection header received
0927 18:40:08.885065 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.885069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.885115 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.885164 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.885950 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.886678 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.887701 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.888044 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.888163 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.889769 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.889773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.889775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.889802 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.891299 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.891757 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.891760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.891762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.891845 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.892186 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.892189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.892191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.892400 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.892517 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.892861 (+ 344us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.892984 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260927 18:40:08.893441 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.893570 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:40:08.898030 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:08.882157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.6.4.62:45955)
0927 18:40:08.882262 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:08.882267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:08.886197 (+ 3930us) server_negotiation.cc:407] Connection header received
0927 18:40:08.886364 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:08.886367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:08.886433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:08.886494 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:08.887255 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.888030 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.892476 (+ 4446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:08.892824 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:08.892892 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:08.894622 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:08.894626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:08.894629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:08.894659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:08.896238 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.896697 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.896700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.896702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.896787 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:08.897090 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:08.897093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:08.897095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:08.897299 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:08.897395 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:08.897757 (+ 362us) server_negotiation.cc:299] Negotiation successful
0927 18:40:08.897877 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:40:08.902458 19689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.902609 19689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:40:08.903033 19689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:08.903177 19689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.904151 19689 leader_election.cc:288] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd570cf9319a4c1090ee7c2fb3048470 (127.6.4.61:38921), f14e07b9e57041aaa5d76d07cce4e428 (127.6.4.60:42221)
I20260927 18:40:08.904609 19652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1192ec60fcce4d95901c7590d6df45d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f14e07b9e57041aaa5d76d07cce4e428" is_pre_election: true
I20260927 18:40:08.904613 19576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1192ec60fcce4d95901c7590d6df45d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd570cf9319a4c1090ee7c2fb3048470" is_pre_election: true
I20260927 18:40:08.904881 19652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1192ec60fcce4d95901c7590d6df45d8 in term 0.
I20260927 18:40:08.904878 19576 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1192ec60fcce4d95901c7590d6df45d8 in term 0.
I20260927 18:40:08.905324 19476 leader_election.cc:302] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1192ec60fcce4d95901c7590d6df45d8, f14e07b9e57041aaa5d76d07cce4e428; no voters:
I20260927 18:40:08.910439 19689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:08.910599 19689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:08.910689 19689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:08.912019 19689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.912701 19576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1192ec60fcce4d95901c7590d6df45d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd570cf9319a4c1090ee7c2fb3048470"
I20260927 18:40:08.912818 19576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:08.912961 19689 leader_election.cc:288] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [CANDIDATE]: Term 1 election: Requested vote from peers cd570cf9319a4c1090ee7c2fb3048470 (127.6.4.61:38921), f14e07b9e57041aaa5d76d07cce4e428 (127.6.4.60:42221)
I20260927 18:40:08.912968 19652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1192ec60fcce4d95901c7590d6df45d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f14e07b9e57041aaa5d76d07cce4e428"
I20260927 18:40:08.913075 19652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:08.914089 19576 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1192ec60fcce4d95901c7590d6df45d8 in term 1.
I20260927 18:40:08.914296 19652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1192ec60fcce4d95901c7590d6df45d8 in term 1.
I20260927 18:40:08.914407 19476 leader_election.cc:302] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1192ec60fcce4d95901c7590d6df45d8, cd570cf9319a4c1090ee7c2fb3048470; no voters:
I20260927 18:40:08.914693 19689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:08.914914 19689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [term 1 LEADER]: Becoming Leader. State: Replica: 1192ec60fcce4d95901c7590d6df45d8, State: Running, Role: LEADER
I20260927 18:40:08.915254 19689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } }
I20260927 18:40:08.919762 19689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1192ec60fcce4d95901c7590d6df45d8. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.919872 19689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:08.920279 19737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:08.921206 19737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:08.926519 19652 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [term 1 FOLLOWER]: Refusing update from remote peer 1192ec60fcce4d95901c7590d6df45d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:08.927310 19735 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:08.933668 19705 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1192ec60fcce4d95901c7590d6df45d8. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.933802 19705 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:40:08.934803 19576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [term 1 FOLLOWER]: Refusing update from remote peer 1192ec60fcce4d95901c7590d6df45d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:08.935361 19735 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:08.938905 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1192ec60fcce4d95901c7590d6df45d8. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.939024 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:40:08.945330 19735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.945465 19735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:08.946462 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.946585 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:40:08.946907 19738 mvcc.cc:205] Tried to move back new op lower bound from 7334028938956693504 to 7334028938927841280. Current Snapshot: MvccSnapshot[applied={T|T < 7334028938956693504}]
I20260927 18:40:08.948532 19741 mvcc.cc:205] Tried to move back new op lower bound from 7334028938956693504 to 7334028938927841280. Current Snapshot: MvccSnapshot[applied={T|T < 7334028938956693504}]
I20260927 18:40:08.949191 19689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.949317 19689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:08.950346 19705 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1192ec60fcce4d95901c7590d6df45d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1192ec60fcce4d95901c7590d6df45d8" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45955 } } peers { permanent_uuid: "cd570cf9319a4c1090ee7c2fb3048470" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 38921 } } peers { permanent_uuid: "f14e07b9e57041aaa5d76d07cce4e428" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 42221 } } }
I20260927 18:40:08.950475 19705 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:40:08.956456 19737 catalog_manager.cc:1347] Generated new cluster ID: 742773d71c1b48489f5c37f0cd04edb7
I20260927 18:40:08.956534 19737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:08.972827 19737 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:40:08.974320 19737 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:09.001111 19737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1192ec60fcce4d95901c7590d6df45d8: Generated new TSK 0
I20260927 18:40:09.001952 19737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:40:09.882227 19729 catalog_manager.cc:1259] Loaded cluster ID: 742773d71c1b48489f5c37f0cd04edb7
I20260927 18:40:09.882314 19729 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: loading cluster ID for follower catalog manager: success
I20260927 18:40:09.884117 19729 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: acquiring CA information for follower catalog manager: success
I20260927 18:40:09.885378 19729 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f14e07b9e57041aaa5d76d07cce4e428: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:40:09.892760 19717 catalog_manager.cc:1259] Loaded cluster ID: 742773d71c1b48489f5c37f0cd04edb7
I20260927 18:40:09.892830 19717 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: loading cluster ID for follower catalog manager: success
I20260927 18:40:09.894426 19717 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: acquiring CA information for follower catalog manager: success
I20260927 18:40:09.895550 19717 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd570cf9319a4c1090ee7c2fb3048470: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:40:10.072638 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19459
I20260927 18:40:10.078389 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19534
I20260927 18:40:10.083134 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19609
2026-09-27T18:40:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2938 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:40:10 dist-test-slave-bzzj krb5kdc[19755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:40:10 dist-test-slave-bzzj krb5kdc[19755](info): set up 2 sockets
Sep 27 18:40:10 dist-test-slave-bzzj krb5kdc[19755](info): commencing operation
krb5kdc: starting...
W20260927 18:40:11.032016 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:40:11 dist-test-slave-bzzj krb5kdc[19755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:40:11.144258 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:41507
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:40057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:41507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:11.255836 19768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:11.256160 19768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:11.256222 19768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:11.260342 19768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:11.260414 19768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:11.260442 19768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:11.260462 19768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:11.260481 19768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:11.265928 19768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:40057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:41507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:41507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:11.267071 19768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:11.268179 19768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:11.276676 19776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:11.276718 19774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:11.276708 19773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:11.276778 19768 server_base.cc:1034] running on GCE node
I20260927 18:40:11.277565 19768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:11.278707 19768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:11.279866 19768 hybrid_clock.cc:648] HybridClock initialized: now 1790534411279844 us; error 33 us; skew 500 ppm
Sep 27 18:40:11 dist-test-slave-bzzj krb5kdc[19755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534411, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:11.282553 19768 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:11.283470 19768 webserver.cc:460] Webserver started at http://127.6.4.62:34417/ using document root <none> and password file <none>
I20260927 18:40:11.284032 19768 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:11.284117 19768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:11.284327 19768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:11.286473 19768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "9e840d1ce1c14c488ecaec3a85770ef2"
format_stamp: "Formatted at 2026-09-27 18:40:11 on dist-test-slave-bzzj"
server_key: "34a3ea49f5ae6177f4fe72582cdb06e7"
server_key_iv: "55ceeffa20c902290405294d65909e76"
server_key_version: "encryptionkey@0"
I20260927 18:40:11.287048 19768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "9e840d1ce1c14c488ecaec3a85770ef2"
format_stamp: "Formatted at 2026-09-27 18:40:11 on dist-test-slave-bzzj"
server_key: "34a3ea49f5ae6177f4fe72582cdb06e7"
server_key_iv: "55ceeffa20c902290405294d65909e76"
server_key_version: "encryptionkey@0"
I20260927 18:40:11.290853 19768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260927 18:40:11.293393 19783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:11.294745 19768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:40:11.294870 19768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "9e840d1ce1c14c488ecaec3a85770ef2"
format_stamp: "Formatted at 2026-09-27 18:40:11 on dist-test-slave-bzzj"
server_key: "34a3ea49f5ae6177f4fe72582cdb06e7"
server_key_iv: "55ceeffa20c902290405294d65909e76"
server_key_version: "encryptionkey@0"
I20260927 18:40:11.294986 19768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:11.304209 19768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:11.307133 19768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:11.307317 19768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:11.314210 19768 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:41507
I20260927 18:40:11.314236 19835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:41507 every 8 connection(s)
I20260927 18:40:11.315196 19768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:11.318046 19836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:11.320456 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19768
I20260927 18:40:11.320574 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:40:11.320832 6160 external_mini_cluster.cc:1428] Setting key 1e89c063df844b5dded4587206f12ccd
I20260927 18:40:11.324119 19836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2: Bootstrap starting.
I20260927 18:40:11.326711 19836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:11.327500 19836 log.cc:824] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:40:11 dist-test-slave-bzzj krb5kdc[19755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:11.329860 19836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2: No bootstrap required, opened a new log
I20260927 18:40:11.333786 19836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41507 } }
I20260927 18:40:11.334121 19836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:11.334187 19836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e840d1ce1c14c488ecaec3a85770ef2, State: Initialized, Role: FOLLOWER
I20260927 18:40:11.334637 19836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41507 } }
I20260927 18:40:11.334770 19836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:11.334836 19836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:11.334931 19836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:11.335024 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:11.322115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.6.4.62:41507)
0927 18:40:11.322527 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:11.322543 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:11.322578 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:40:11.323238 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:11.323256 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:11.323604 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:11.323893 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:11.324824 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:11.325821 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:11.326961 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:11.327433 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:11.327552 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:11.330343 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:11.330348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:11.330363 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:11.330390 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:11.331952 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:11.332458 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:11.332463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:11.332467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:11.332583 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:11.332864 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:11.332867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:11.332868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:11.333328 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:11.333489 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:11.333776 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 18:40:11.334119 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
I20260927 18:40:11.336138 19836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41507 } }
I20260927 18:40:11.336477 19836 leader_election.cc:302] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e840d1ce1c14c488ecaec3a85770ef2; no voters:
I20260927 18:40:11.336776 19836 leader_election.cc:288] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:40:11.336892 19841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:11.337111 19841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [term 1 LEADER]: Becoming Leader. State: Replica: 9e840d1ce1c14c488ecaec3a85770ef2, State: Running, Role: LEADER
I20260927 18:40:11.337456 19841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41507 } }
I20260927 18:40:11.338083 19836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:11.338335 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:40:11.339246 19842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e840d1ce1c14c488ecaec3a85770ef2. Latest consensus state: current_term: 1 leader_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41507 } } }
I20260927 18:40:11.339382 19842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:11.339246 19843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e840d1ce1c14c488ecaec3a85770ef2" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 41507 } } }
I20260927 18:40:11.339493 19843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:11.340008 19850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:11.342808 19850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:11.348229 19850 catalog_manager.cc:1347] Generated new cluster ID: 62ca36777feb45959f868887419255e3
I20260927 18:40:11.348310 19850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:11.373184 19850 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:40:11.374581 19850 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:11.382614 19850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e840d1ce1c14c488ecaec3a85770ef2: Generated new TSK 0
I20260927 18:40:11.383496 19850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:40:11.394217 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19768
2026-09-27T18:40:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1309 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:40:11 dist-test-slave-bzzj krb5kdc[19863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:40:11 dist-test-slave-bzzj krb5kdc[19863](info): set up 2 sockets
Sep 27 18:40:11 dist-test-slave-bzzj krb5kdc[19863](info): commencing operation
krb5kdc: starting...
W20260927 18:40:12.336159 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:40:12 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:40:12.451498 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40247
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:12.561164 19876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:12.561437 19876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:12.561494 19876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:12.565493 19876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:12.565562 19876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:12.565588 19876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:12.565609 19876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:12.565635 19876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:12.570883 19876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:12.571943 19876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:12.573050 19876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:40:12.581540 19876 server_base.cc:1034] running on GCE node
W20260927 18:40:12.581490 19884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:12.581488 19881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:12.581497 19882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:12.582377 19876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:12.583464 19876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:12.584614 19876 hybrid_clock.cc:648] HybridClock initialized: now 1790534412584599 us; error 30 us; skew 500 ppm
Sep 27 18:40:12 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:12.591351 19876 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:12.592300 19876 webserver.cc:460] Webserver started at http://127.6.4.62:41535/ using document root <none> and password file <none>
I20260927 18:40:12.592839 19876 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:12.592905 19876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:12.593124 19876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:12.595196 19876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "ce9c80012aca43d1b5c80aff8033e0cc"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "e2035b264c3c4f3bf34d6d714ee1ca46"
server_key_iv: "9145bf406a04a818d1ef9fb6d5f8154d"
server_key_version: "encryptionkey@0"
I20260927 18:40:12.595734 19876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "ce9c80012aca43d1b5c80aff8033e0cc"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "e2035b264c3c4f3bf34d6d714ee1ca46"
server_key_iv: "9145bf406a04a818d1ef9fb6d5f8154d"
server_key_version: "encryptionkey@0"
I20260927 18:40:12.599406 19876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:40:12.601812 19891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:12.602998 19876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:40:12.603118 19876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "ce9c80012aca43d1b5c80aff8033e0cc"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "e2035b264c3c4f3bf34d6d714ee1ca46"
server_key_iv: "9145bf406a04a818d1ef9fb6d5f8154d"
server_key_version: "encryptionkey@0"
I20260927 18:40:12.603209 19876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:12.615670 19876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:12.618752 19876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:12.618932 19876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:12.625644 19876 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40247
I20260927 18:40:12.625897 19943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40247 every 8 connection(s)
I20260927 18:40:12.626603 19876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:12.627345 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19876
I20260927 18:40:12.627465 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:40:12.627761 6160 external_mini_cluster.cc:1428] Setting key c829710c66166511d967475b64cbe06c
I20260927 18:40:12.629907 19944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:12.635743 19944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap starting.
Sep 27 18:40:12 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:12.638401 19944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:12.639154 19944 log.cc:824] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:12.641338 19944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: No bootstrap required, opened a new log
I20260927 18:40:12.641749 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:12.629247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.6.4.62:40247)
0927 18:40:12.629681 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:12.629695 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:12.629731 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:40:12.630407 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:12.630426 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:12.630775 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:12.631082 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:12.631842 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:12.632870 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:12.633739 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:12.634459 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:12.634620 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:12.637279 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:12.637284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:12.637298 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:12.637325 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:12.638892 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:12.639367 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:12.639372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:12.639377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:12.639488 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:12.639757 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:12.639761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:12.639763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:12.640255 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:12.640396 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:12.640693 (+ 297us) server_negotiation.cc:299] Negotiation successful
0927 18:40:12.640970 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260927 18:40:12.644783 19944 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:12.645012 19944 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:12.645068 19944 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9c80012aca43d1b5c80aff8033e0cc, State: Initialized, Role: FOLLOWER
I20260927 18:40:12.645463 19944 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:12.645596 19944 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:12.645656 19944 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:12.645752 19944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:12.646607 19944 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:12.646916 19944 leader_election.cc:302] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9c80012aca43d1b5c80aff8033e0cc; no voters:
I20260927 18:40:12.647198 19944 leader_election.cc:288] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:40:12.647303 19949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:12.647517 19949 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 LEADER]: Becoming Leader. State: Replica: ce9c80012aca43d1b5c80aff8033e0cc, State: Running, Role: LEADER
I20260927 18:40:12.647818 19949 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:12.648334 19944 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:12.649375 19950 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } } }
I20260927 18:40:12.649497 19950 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:12.649827 19951 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9c80012aca43d1b5c80aff8033e0cc. Latest consensus state: current_term: 1 leader_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } } }
I20260927 18:40:12.649926 19951 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: This master's current role is: LEADER
W20260927 18:40:12.652904 19964 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:40:12.652966 19964 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:40:12.653030 19958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:12.653666 19958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:12.658530 19958 catalog_manager.cc:1347] Generated new cluster ID: 4759ba93e425477f867bdfd114c53797
I20260927 18:40:12.658591 19958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:12.676314 19958 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:40:12.677614 19958 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:12.683930 19958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Generated new TSK 0
I20260927 18:40:12.684795 19958 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:40:12.715356 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40247
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:12.824942 19970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:12.825259 19970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:12.825325 19970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:12.829416 19970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:12.829483 19970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:12.829562 19970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:12.834913 19970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:12.835975 19970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:12.837036 19970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:12.846335 19975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:12.846510 19970 server_base.cc:1034] running on GCE node
W20260927 18:40:12.846338 19976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:12.846357 19978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:12.847048 19970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:12.847685 19970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:12.848835 19970 hybrid_clock.cc:648] HybridClock initialized: now 1790534412848816 us; error 31 us; skew 500 ppm
Sep 27 18:40:12 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:12.851531 19970 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:12.852416 19970 webserver.cc:460] Webserver started at http://127.6.4.1:41925/ using document root <none> and password file <none>
I20260927 18:40:12.852952 19970 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:12.853016 19970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:12.853199 19970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:12.855214 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "66354bdec6b216708df5315250ed4db3"
server_key_iv: "5ab8ce4e92b26832a18a27bb27102c66"
server_key_version: "encryptionkey@0"
I20260927 18:40:12.855753 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "66354bdec6b216708df5315250ed4db3"
server_key_iv: "5ab8ce4e92b26832a18a27bb27102c66"
server_key_version: "encryptionkey@0"
I20260927 18:40:12.859298 19970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:40:12.861644 19985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:12.862830 19970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:40:12.862936 19970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "66354bdec6b216708df5315250ed4db3"
server_key_iv: "5ab8ce4e92b26832a18a27bb27102c66"
server_key_version: "encryptionkey@0"
I20260927 18:40:12.863029 19970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:12.887424 19970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:12.890604 19970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:12.890779 19970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:12.891372 19970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:12.892248 19970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:12.892303 19970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:12.892365 19970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:12.892390 19970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:12.900638 19970 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:44297
I20260927 18:40:12.900666 20098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:44297 every 8 connection(s)
I20260927 18:40:12.901566 19970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:40:12 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534412, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:12.911859 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 19970
I20260927 18:40:12.911985 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:40:12.912344 6160 external_mini_cluster.cc:1428] Setting key 4c1f61f4ec983c5aa7df1b787ac76799
I20260927 18:40:12.914762 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:12.903319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:43621 (local address 127.6.4.62:40247)
0927 18:40:12.903449 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:12.903452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:12.904303 (+ 851us) server_negotiation.cc:407] Connection header received
0927 18:40:12.905556 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:12.905561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:12.905619 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:12.905692 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:12.906917 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:12.907476 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:12.908649 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:12.908891 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:12.908981 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:12.911236 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:12.911239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:12.911241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:12.911267 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:12.912718 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:12.913214 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:12.913217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:12.913218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:12.913300 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:12.913653 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:12.913656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:12.913658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:12.913869 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:12.913976 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:12.914484 (+ 508us) server_negotiation.cc:299] Negotiation successful
0927 18:40:12.914616 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:40:12.915539 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:12.903699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:43621)
0927 18:40:12.904172 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:12.904215 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:12.905347 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:12.905878 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:12.905886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:12.906284 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:12.906774 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:12.906787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:12.907598 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:12.907601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:12.908538 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:12.908548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:12.908976 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:12.909043 (+ 67us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:12.909201 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:12.909699 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:12.909714 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:12.911077 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:12.912830 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:12.912836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:12.912852 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:12.913134 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:12.913382 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:12.913385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:12.913387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:12.913565 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:12.913995 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:12.914001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:12.914256 (+ 255us) client_negotiation.cc:769] Sending connection context
0927 18:40:12.914411 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 18:40:12.914677 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
I20260927 18:40:12.916757 20099 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40247
I20260927 18:40:12.917008 20099 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:12.917575 20099 heartbeater.cc:511] Master 127.6.4.62:40247 requested a full tablet report, sending...
I20260927 18:40:12.919167 19908 ts_manager.cc:194] Registered new tserver with Master: a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:12.920359 19908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:43621
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:40:12.937237 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40247
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:13.044914 20104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:13.045161 20104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:13.045202 20104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:13.049218 20104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:13.049274 20104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:13.049332 20104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:13.054569 20104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:13.055567 20104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:13.056555 20104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:40:13.065845 20104 server_base.cc:1034] running on GCE node
W20260927 18:40:13.065799 20112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:13.065793 20109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:13.065793 20110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:13.066483 20104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:13.067098 20104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:13.068250 20104 hybrid_clock.cc:648] HybridClock initialized: now 1790534413068232 us; error 32 us; skew 500 ppm
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:13.070858 20104 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:13.071694 20104 webserver.cc:460] Webserver started at http://127.6.4.2:44323/ using document root <none> and password file <none>
I20260927 18:40:13.072271 20104 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:13.072335 20104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:13.072507 20104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:13.074501 20104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "cff6f8e40df74ddd8c05069c54e86feb"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "fe797c7a653b4a652e2578b1a200d161"
server_key_iv: "14d10e1ccf51ee513d2fa48aec7e8a67"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.075008 20104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "cff6f8e40df74ddd8c05069c54e86feb"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "fe797c7a653b4a652e2578b1a200d161"
server_key_iv: "14d10e1ccf51ee513d2fa48aec7e8a67"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.078555 20104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 18:40:13.080850 20119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.082036 20104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:40:13.082135 20104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "cff6f8e40df74ddd8c05069c54e86feb"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "fe797c7a653b4a652e2578b1a200d161"
server_key_iv: "14d10e1ccf51ee513d2fa48aec7e8a67"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.082217 20104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:13.109846 20104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:13.112717 20104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:13.112881 20104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:13.113435 20104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:13.114266 20104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:13.114310 20104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.114348 20104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:13.114368 20104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.122623 20104 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36509
I20260927 18:40:13.122649 20232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36509 every 8 connection(s)
I20260927 18:40:13.123536 20104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:13.133234 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20104
I20260927 18:40:13.133354 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:40:13.133673 6160 external_mini_cluster.cc:1428] Setting key d45356504f11604f040f529b882afb4b
I20260927 18:40:13.136709 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.125330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:33815 (local address 127.6.4.62:40247)
0927 18:40:13.125482 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.125487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.126327 (+ 840us) server_negotiation.cc:407] Connection header received
0927 18:40:13.127523 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.127526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.127573 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.127642 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:13.128886 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.129417 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.130489 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.130698 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.130764 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.133221 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:13.133226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:13.133228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:13.133259 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:13.134670 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.135224 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.135228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.135229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.135319 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.135630 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.135633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.135635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.135839 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:13.135937 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.136428 (+ 491us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.136553 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:40:13.137701 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.125742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:33815)
0927 18:40:13.126177 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.126220 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.127327 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.127791 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.127798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.128204 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:13.128759 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.128774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.129521 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.129523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.130400 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.130408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.130762 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.130806 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.130949 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.131457 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:13.131472 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:13.133024 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:13.134775 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.134782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.134796 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.135136 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.135403 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.135405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.135406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.135534 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.135925 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:13.135930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:13.136211 (+ 281us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.136370 (+ 159us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.136643 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":114,"threads_started":1}
I20260927 18:40:13.138856 20233 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40247
I20260927 18:40:13.139088 20233 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:13.139595 20233 heartbeater.cc:511] Master 127.6.4.62:40247 requested a full tablet report, sending...
I20260927 18:40:13.140817 19908 ts_manager.cc:194] Registered new tserver with Master: cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509)
I20260927 18:40:13.141398 19908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:33815
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:40:13.159412 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40247
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:13.268407 20238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:13.268672 20238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:13.268731 20238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:13.272728 20238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:13.272790 20238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:13.272871 20238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:13.278151 20238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:13.279222 20238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:13.280277 20238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:13.289662 20243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:13.289666 20246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:13.289700 20238 server_base.cc:1034] running on GCE node
W20260927 18:40:13.289746 20244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:13.290254 20238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:13.290884 20238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:13.292064 20238 hybrid_clock.cc:648] HybridClock initialized: now 1790534413292044 us; error 34 us; skew 500 ppm
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:13.294679 20238 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:13.295531 20238 webserver.cc:460] Webserver started at http://127.6.4.3:45741/ using document root <none> and password file <none>
I20260927 18:40:13.296065 20238 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:13.296154 20238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:13.296356 20238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:13.298331 20238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "acc9732d8f1c4844ae5d6a68b911c89a"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "3c968013ecae23518356bcb348fc0062"
server_key_iv: "1b68fb2a80ab1b07b4e280fddddaa863"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.298844 20238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "acc9732d8f1c4844ae5d6a68b911c89a"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "3c968013ecae23518356bcb348fc0062"
server_key_iv: "1b68fb2a80ab1b07b4e280fddddaa863"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.302417 20238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 18:40:13.304807 20253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.306083 20238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:13.306180 20238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "acc9732d8f1c4844ae5d6a68b911c89a"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "3c968013ecae23518356bcb348fc0062"
server_key_iv: "1b68fb2a80ab1b07b4e280fddddaa863"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.306268 20238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:13.321391 20238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:13.324281 20238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:13.324450 20238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:13.324988 20238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:13.325781 20238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:13.325830 20238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.325875 20238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:13.325892 20238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.333846 20238 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37401
I20260927 18:40:13.333936 20366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37401 every 8 connection(s)
I20260927 18:40:13.334744 20238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:13.335279 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20238
I20260927 18:40:13.335381 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:40:13.335665 6160 external_mini_cluster.cc:1428] Setting key 16bcaa39c684097ba97c969962d62a48
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:13.347823 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.336545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:46447 (local address 127.6.4.62:40247)
0927 18:40:13.336671 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.336676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.337666 (+ 990us) server_negotiation.cc:407] Connection header received
0927 18:40:13.339002 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.339005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.339062 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.339134 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:13.340355 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.340895 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.342017 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.342236 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.342322 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.344715 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:13.344718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:13.344720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:13.344746 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:13.345855 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.346343 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.346347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.346349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.346428 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.346734 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.346737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.346739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.346941 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:13.347027 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.347519 (+ 492us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.347644 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:40:13.348847 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.337046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:46447)
0927 18:40:13.337535 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.337577 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.338800 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.339324 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.339332 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.339690 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:13.340225 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.340241 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.341009 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.341013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.341901 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.341911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.342319 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.342381 (+ 62us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.342550 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.343092 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:13.343108 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:13.344556 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:13.345989 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.345995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.346010 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.346252 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.346507 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.346510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.346512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.346649 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.347043 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:13.347050 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:13.347289 (+ 239us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.347461 (+ 172us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.347781 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":122,"threads_started":1}
I20260927 18:40:13.349928 20367 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40247
I20260927 18:40:13.350142 20367 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:13.350653 20367 heartbeater.cc:511] Master 127.6.4.62:40247 requested a full tablet report, sending...
I20260927 18:40:13.351570 19908 ts_manager.cc:194] Registered new tserver with Master: acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401)
I20260927 18:40:13.352144 19908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:46447
I20260927 18:40:13.361629 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:13.370412 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.363523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.6.4.62:40247)
0927 18:40:13.363646 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.363650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.363790 (+ 140us) server_negotiation.cc:407] Connection header received
0927 18:40:13.363918 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.363922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.363969 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.364031 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:13.364750 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.365279 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.366088 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.366341 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.366425 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.367660 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:13.367664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:13.367667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:13.367695 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:13.368872 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.369249 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.369252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.369254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.369332 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.369566 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.369569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.369571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.369803 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:13.369920 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.370149 (+ 229us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.370258 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:40:13.373196 19908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:40:13.376396 19908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:40:13.390681 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.385698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.6.4.2:36509)
0927 18:40:13.386476 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.386487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.386506 (+ 19us) server_negotiation.cc:407] Connection header received
0927 18:40:13.386572 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.386576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.386786 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.386917 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:13.387117 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.387891 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.389230 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.390024 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.390249 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.390394 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.390450 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.390538 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":97,"threads_started":1}
I20260927 18:40:13.390676 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.384939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.0.0.1:42786)
0927 18:40:13.385882 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.385894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.385981 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.386716 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.386719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.386738 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:13.386820 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.386826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.388090 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.388124 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.389278 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.389286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.390088 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.390091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.390194 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.390247 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.390344 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.390459 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":29,"thread_start_us":86,"threads_started":1}
I20260927 18:40:13.390789 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.385346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.6.4.1:44297)
0927 18:40:13.385635 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.385648 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.386299 (+ 651us) server_negotiation.cc:407] Connection header received
0927 18:40:13.386378 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.386383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.386577 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.386722 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:13.386941 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.387941 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.389393 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.389975 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.390079 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.390231 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.390540 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.390651 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
I20260927 18:40:13.390823 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.384621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37401 (local address 127.0.0.1:48508)
0927 18:40:13.385050 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.385105 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.385240 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.386122 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.386128 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.386150 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:13.386295 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.386303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.387740 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.387743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.389227 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.389237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.390179 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.390183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.390555 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.390576 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.390636 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.390694 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":326,"thread_start_us":89,"threads_started":1}
I20260927 18:40:13.391083 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.385372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36509 (local address 127.0.0.1:60954)
0927 18:40:13.386155 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.386167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.386239 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.386898 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.386901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.386918 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:13.387016 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.387022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.388025 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.388029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.389120 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.389129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.390179 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.390183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.390348 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.390368 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.390416 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.390465 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":112,"thread_start_us":82,"threads_started":1}
I20260927 18:40:13.391952 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.384740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.6.4.3:37401)
0927 18:40:13.385537 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.385566 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.385590 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:40:13.385670 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.385676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.385904 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.386126 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:13.386424 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.387613 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.389340 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.390068 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.390298 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.390409 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.391720 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.391821 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":167,"threads_started":1}
I20260927 18:40:13.393808 20033 tablet_service.cc:1467] Processing CreateTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f (DEFAULT_TABLE table=test-table [id=7f817174067c4524bc62b8bf7224549e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:13.394279 20301 tablet_service.cc:1467] Processing CreateTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f (DEFAULT_TABLE table=test-table [id=7f817174067c4524bc62b8bf7224549e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:13.394336 20167 tablet_service.cc:1467] Processing CreateTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f (DEFAULT_TABLE table=test-table [id=7f817174067c4524bc62b8bf7224549e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:13.395020 20033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f6fa953c44a47edbb33e828bcbbf57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:13.395151 20301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f6fa953c44a47edbb33e828bcbbf57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:13.395256 20167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f6fa953c44a47edbb33e828bcbbf57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:13.401907 20384 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Bootstrap starting.
I20260927 18:40:13.402154 20386 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Bootstrap starting.
I20260927 18:40:13.404378 20385 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap starting.
I20260927 18:40:13.404918 20386 tablet_bootstrap.cc:654] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:13.404918 20384 tablet_bootstrap.cc:654] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:13.405817 20384 log.cc:824] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:13.405817 20386 log.cc:824] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:13.406158 20385 tablet_bootstrap.cc:654] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:13.407403 20385 log.cc:824] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:13.407956 20386 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: No bootstrap required, opened a new log
I20260927 18:40:13.407986 20384 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: No bootstrap required, opened a new log
I20260927 18:40:13.408175 20386 ts_tablet_manager.cc:1397] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260927 18:40:13.408197 20384 ts_tablet_manager.cc:1397] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:40:13.409227 20385 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: No bootstrap required, opened a new log
I20260927 18:40:13.409389 20385 ts_tablet_manager.cc:1397] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:40:13.411266 20384 raft_consensus.cc:348] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.411490 20384 raft_consensus.cc:374] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:13.411561 20384 raft_consensus.cc:729] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc9732d8f1c4844ae5d6a68b911c89a, State: Initialized, Role: FOLLOWER
I20260927 18:40:13.411870 20386 raft_consensus.cc:348] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.412147 20386 raft_consensus.cc:374] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:13.412144 20384 consensus_queue.cc:260] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.412248 20386 raft_consensus.cc:729] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff6f8e40df74ddd8c05069c54e86feb, State: Initialized, Role: FOLLOWER
I20260927 18:40:13.412674 20386 consensus_queue.cc:260] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.412839 20367 heartbeater.cc:503] Master 127.6.4.62:40247 was elected leader, sending a full tablet report...
I20260927 18:40:13.413170 20384 ts_tablet_manager.cc:1428] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:40:13.413290 20233 heartbeater.cc:503] Master 127.6.4.62:40247 was elected leader, sending a full tablet report...
I20260927 18:40:13.413573 20386 ts_tablet_manager.cc:1428] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260927 18:40:13.414319 20385 raft_consensus.cc:348] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.414558 20385 raft_consensus.cc:374] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:13.414655 20385 raft_consensus.cc:729] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2b2c266f44eed8f7cb21b76ac8eba, State: Initialized, Role: FOLLOWER
I20260927 18:40:13.415104 20385 consensus_queue.cc:260] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.415720 20099 heartbeater.cc:503] Master 127.6.4.62:40247 was elected leader, sending a full tablet report...
I20260927 18:40:13.415941 20385 ts_tablet_manager.cc:1428] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 18:40:13.413278 20390 raft_consensus.cc:482] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:13.416610 20390 raft_consensus.cc:504] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.418205 20390 leader_election.cc:288] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509), a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:13.422999 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.418716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:40723 (local address 127.6.4.1:44297)
0927 18:40:13.418816 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.418821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.419006 (+ 185us) server_negotiation.cc:407] Connection header received
0927 18:40:13.419127 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.419130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.419170 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.419233 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:13.419523 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.420313 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.421916 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.422486 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.422611 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.422672 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.422816 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.422887 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:40:13.423110 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.418348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36509 (local address 127.6.4.3:44451)
0927 18:40:13.418445 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.418457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.418518 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.419222 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.419225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.419242 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:13.419432 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.419437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.420415 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.420417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.421792 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.421802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.422735 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.422738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.422844 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.422863 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.422920 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.422975 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":107,"spinlock_wait_cycles":384}
I20260927 18:40:13.423255 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.418933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:44451 (local address 127.6.4.2:36509)
0927 18:40:13.419027 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.419031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.419051 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:40:13.419092 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.419095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.419137 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.419204 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:13.419523 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.420313 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.422066 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.422642 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.422731 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.422781 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.423075 (+ 294us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.423153 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:40:13.423023 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.418642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.6.4.3:40723)
0927 18:40:13.418934 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.418946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.419037 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.419242 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.419244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.419255 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:13.419428 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.419436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.420401 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.420404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.421792 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.421802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.422612 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.422615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.422695 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.422722 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.422795 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.422869 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":118,"thread_start_us":86,"threads_started":1}
I20260927 18:40:13.423688 20053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f6fa953c44a47edbb33e828bcbbf57f" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" is_pre_election: true
I20260927 18:40:13.423691 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f6fa953c44a47edbb33e828bcbbf57f" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff6f8e40df74ddd8c05069c54e86feb" is_pre_election: true
I20260927 18:40:13.423908 20053 raft_consensus.cc:2454] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 0.
I20260927 18:40:13.423918 20187 raft_consensus.cc:2454] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 0.
I20260927 18:40:13.424312 20257 leader_election.cc:302] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a3a2b2c266f44eed8f7cb21b76ac8eba, acc9732d8f1c4844ae5d6a68b911c89a, cff6f8e40df74ddd8c05069c54e86feb; no voters:
I20260927 18:40:13.424574 20390 raft_consensus.cc:2781] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:13.424645 20390 raft_consensus.cc:482] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:13.424676 20390 raft_consensus.cc:3037] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:13.425634 20390 raft_consensus.cc:504] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.425964 20390 leader_election.cc:288] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 election: Requested vote from peers cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509), a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:13.426226 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f6fa953c44a47edbb33e828bcbbf57f" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff6f8e40df74ddd8c05069c54e86feb"
I20260927 18:40:13.426254 20053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f6fa953c44a47edbb33e828bcbbf57f" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
I20260927 18:40:13.426321 20187 raft_consensus.cc:3037] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:13.426348 20053 raft_consensus.cc:3037] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:13.427290 20053 raft_consensus.cc:2454] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 1.
I20260927 18:40:13.427294 20187 raft_consensus.cc:2454] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 1.
I20260927 18:40:13.427517 20257 leader_election.cc:302] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a2b2c266f44eed8f7cb21b76ac8eba, acc9732d8f1c4844ae5d6a68b911c89a; no voters:
I20260927 18:40:13.427664 20390 raft_consensus.cc:2781] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:13.427755 20390 raft_consensus.cc:686] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 1 LEADER]: Becoming Leader. State: Replica: acc9732d8f1c4844ae5d6a68b911c89a, State: Running, Role: LEADER
I20260927 18:40:13.428030 20390 consensus_queue.cc:237] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.431046 19907 catalog_manager.cc:5495] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a reported cstate change: term changed from 0 to 1, leader changed from <none> to acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3). New cstate: current_term: 1 leader_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } health_report { overall_health: UNKNOWN } } }
I20260927 18:40:13.440475 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.437449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.6.4.3:37401)
0927 18:40:13.437558 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.437561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.437573 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:40:13.437666 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.437668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.437716 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.437794 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:13.437974 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.438455 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.439225 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.439435 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.439505 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.439709 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:13.440168 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:13.440257 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.440296 (+ 39us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.440345 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 18:40:13.446223 20187 raft_consensus.cc:1261] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Refusing update from remote peer acc9732d8f1c4844ae5d6a68b911c89a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:13.446224 20053 raft_consensus.cc:1261] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Refusing update from remote peer acc9732d8f1c4844ae5d6a68b911c89a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:13.446857 20391 consensus_queue.cc:1035] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:13.447031 20390 consensus_queue.cc:1035] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:13.455211 20399 mvcc.cc:205] Tried to move back new op lower bound from 7334028957468667904 to 7334028957404184576. Current Snapshot: MvccSnapshot[applied={T|T < 7334028957468667904}]
I20260927 18:40:13.456050 20398 mvcc.cc:205] Tried to move back new op lower bound from 7334028957468667904 to 7334028957404184576. Current Snapshot: MvccSnapshot[applied={T|T < 7334028957468667904}]
I20260927 18:40:13.464191 19908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:13.464383 19908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:13.466781 19908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Sending DeleteTablet for 3 replicas of tablet 8f6fa953c44a47edbb33e828bcbbf57f
I20260927 18:40:13.467559 20033 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:13 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:42786
I20260927 18:40:13.467589 20167 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:13 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60954
I20260927 18:40:13.467723 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19970
I20260927 18:40:13.468809 20406 tablet_replica.cc:330] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: stopping tablet replica
I20260927 18:40:13.469082 20406 raft_consensus.cc:2229] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:13.469368 20406 raft_consensus.cc:2258] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:13.470224 20301 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:13 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:48508
I20260927 18:40:13.470741 20407 tablet_replica.cc:330] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: stopping tablet replica
I20260927 18:40:13.470824 20406 ts_tablet_manager.cc:1905] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:13.471061 20407 raft_consensus.cc:2229] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:40:13.471498 20407 raft_consensus.cc:2258] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:13.473063 20407 ts_tablet_manager.cc:1905] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:13.473614 20406 ts_tablet_manager.cc:1918] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:13.473739 20406 log.cc:1196] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/8f6fa953c44a47edbb33e828bcbbf57f
I20260927 18:40:13.474100 20406 ts_tablet_manager.cc:1939] T 8f6fa953c44a47edbb33e828bcbbf57f P cff6f8e40df74ddd8c05069c54e86feb: Deleting consensus metadata
I20260927 18:40:13.475729 20407 ts_tablet_manager.cc:1918] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:13.475684 19892 catalog_manager.cc:4841] TS cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509): tablet 8f6fa953c44a47edbb33e828bcbbf57f (table test-table [id=7f817174067c4524bc62b8bf7224549e]) successfully deleted
I20260927 18:40:13.475807 20407 log.cc:1196] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/8f6fa953c44a47edbb33e828bcbbf57f
I20260927 18:40:13.476121 20407 ts_tablet_manager.cc:1939] T 8f6fa953c44a47edbb33e828bcbbf57f P acc9732d8f1c4844ae5d6a68b911c89a: Deleting consensus metadata
W20260927 18:40:13.476336 19895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:44297 (error 108)
I20260927 18:40:13.476591 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 20104
I20260927 18:40:13.477805 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.477523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.0.0.1:42800)
0927 18:40:13.477647 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.477716 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:44297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260927 18:40:13.477929 19895 catalog_manager.cc:4584] TS a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f6fa953c44a47edbb33e828bcbbf57f: Network error: Client connection negotiation failed: client connection to 127.6.4.1:44297: connect: Connection refused (error 111)
I20260927 18:40:13.478785 19892 catalog_manager.cc:4841] TS acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401): tablet 8f6fa953c44a47edbb33e828bcbbf57f (table test-table [id=7f817174067c4524bc62b8bf7224549e]) successfully deleted
I20260927 18:40:13.484074 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 20238
I20260927 18:40:13.488895 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 19876
I20260927 18:40:13.494167 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40247
--webserver_interface=127.6.4.62
--webserver_port=41535
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:13.602212 20409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:13.602479 20409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:13.602535 20409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:13.606767 20409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:13.606846 20409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:13.606878 20409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:13.606899 20409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:13.606917 20409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:13.612380 20409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=41535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:13.613440 20409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:13.614478 20409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:13.626694 20418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:13.626686 20416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:13.626772 20409 server_base.cc:1034] running on GCE node
W20260927 18:40:13.626700 20415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:13.627527 20409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:13.628691 20409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:13.629850 20409 hybrid_clock.cc:648] HybridClock initialized: now 1790534413629828 us; error 33 us; skew 500 ppm
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:13.632536 20409 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:13.633387 20409 webserver.cc:460] Webserver started at http://127.6.4.62:41535/ using document root <none> and password file <none>
I20260927 18:40:13.633958 20409 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:13.634037 20409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:13.637918 20409 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:13.640048 20425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.641244 20409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:13.641357 20409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "ce9c80012aca43d1b5c80aff8033e0cc"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "e2035b264c3c4f3bf34d6d714ee1ca46"
server_key_iv: "9145bf406a04a818d1ef9fb6d5f8154d"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.641942 20409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:13.657251 20409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:13.659991 20409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:13.660182 20409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:13.666818 20409 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40247
I20260927 18:40:13.667076 20477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40247 every 8 connection(s)
I20260927 18:40:13.667891 20409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:13.669499 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20409
I20260927 18:40:13.669875 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:44297
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:40247
--webserver_port=41925
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:13.672761 20478 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:40:13.674486 20478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap starting.
I20260927 18:40:13.691314 20478 log.cc:824] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:13.700698 20478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:13.701064 20478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap complete.
I20260927 18:40:13.704981 20478 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:13.705384 20478 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9c80012aca43d1b5c80aff8033e0cc, State: Initialized, Role: FOLLOWER
I20260927 18:40:13.705891 20478 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:13.706024 20478 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:13.706095 20478 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:13.706192 20478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:40:13.708119 20478 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:13.708461 20478 leader_election.cc:302] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9c80012aca43d1b5c80aff8033e0cc; no voters:
I20260927 18:40:13.708820 20478 leader_election.cc:288] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:40:13.709271 20483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:40:13.709650 20483 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 LEADER]: Becoming Leader. State: Replica: ce9c80012aca43d1b5c80aff8033e0cc, State: Running, Role: LEADER
I20260927 18:40:13.710003 20483 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:13.710162 20478 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:13.711377 20484 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9c80012aca43d1b5c80aff8033e0cc. Latest consensus state: current_term: 2 leader_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } } }
I20260927 18:40:13.711491 20484 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:13.712183 20482 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } } }
I20260927 18:40:13.712275 20482 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:13.714932 20499 catalog_manager.cc:1259] Loaded cluster ID: 4759ba93e425477f867bdfd114c53797
I20260927 18:40:13.714982 20499 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: loading cluster ID for follower catalog manager: success
W20260927 18:40:13.715758 20499 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260927 18:40:13.715817 20499 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260927 18:40:13.715979 20492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:13.717938 20492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:13.718345 20492 catalog_manager.cc:1259] Loaded cluster ID: 4759ba93e425477f867bdfd114c53797
I20260927 18:40:13.718393 20492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:13.719182 20492 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 18:40:13.721901 20492 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 18:40:13.726099 20492 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:13.726768 20492 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Loaded TSK: 0
I20260927 18:40:13.727723 20492 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:40:13.792677 20480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:13.792948 20480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:13.792990 20480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:13.797051 20480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:13.797119 20480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:13.797186 20480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:13.802429 20480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:44297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=41925
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:13.803447 20480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:13.804488 20480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:40:13.815027 20480 server_base.cc:1034] running on GCE node
W20260927 18:40:13.814929 20507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:13.814913 20506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:13.815270 20509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:13.815675 20480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:13.816336 20480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:13.817507 20480 hybrid_clock.cc:648] HybridClock initialized: now 1790534413817476 us; error 45 us; skew 500 ppm
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:13.820235 20480 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:13.821146 20480 webserver.cc:460] Webserver started at http://127.6.4.1:41925/ using document root <none> and password file <none>
I20260927 18:40:13.821671 20480 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:13.821748 20480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:13.825678 20480 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260927 18:40:13.827775 20516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:13.829005 20480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:13.829107 20480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "66354bdec6b216708df5315250ed4db3"
server_key_iv: "5ab8ce4e92b26832a18a27bb27102c66"
server_key_version: "encryptionkey@0"
I20260927 18:40:13.829645 20480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:13.844475 20480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:13.847544 20480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:13.847713 20480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:13.848351 20480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:13.849471 20523 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:40:13.853487 20480 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:40:13.853542 20480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260927 18:40:13.853593 20480 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:40:13.855229 20480 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:40:13.855283 20480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260927 18:40:13.855557 20523 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap starting.
I20260927 18:40:13.865278 20480 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:44297
I20260927 18:40:13.865334 20630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:44297 every 8 connection(s)
I20260927 18:40:13.866472 20480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:13.868963 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20480
I20260927 18:40:13.869449 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:36509
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:40247
--webserver_port=44323
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:13.876221 20523 log.cc:824] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:13.884478 20523 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:13.884922 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.868502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:57563 (local address 127.6.4.62:40247)
0927 18:40:13.871166 (+ 2664us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.871180 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.871217 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:40:13.871443 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.871454 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.871752 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.872072 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:13.873261 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.874249 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.875398 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.876060 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.876230 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.879125 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:13.879131 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:13.879147 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:13.879181 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:13.881206 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.881852 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.881858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.881863 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.882018 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.882403 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.882406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.882408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.882863 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:13.882989 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.883682 (+ 693us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.883991 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2534,"thread_start_us":94,"threads_started":1}
I20260927 18:40:13.884923 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.868570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:57563)
0927 18:40:13.869127 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.869183 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.870162 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.872181 (+ 2019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.872190 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.872608 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:13.873091 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.873107 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.874397 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.874400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.875278 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.875288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.876186 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.876223 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.876415 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.877023 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:13.877045 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:13.878897 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:13.881339 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.881345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.881359 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.881732 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.882120 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.882123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.882125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.882302 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.882981 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:13.882989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:13.883314 (+ 325us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.883541 (+ 227us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.883928 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":127,"threads_started":1}
I20260927 18:40:13.884933 20523 tablet_bootstrap.cc:492] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap complete.
I20260927 18:40:13.886289 20523 ts_tablet_manager.cc:1397] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.012s
I20260927 18:40:13.887908 20631 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40247
I20260927 18:40:13.888176 20631 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:13.888736 20631 heartbeater.cc:511] Master 127.6.4.62:40247 requested a full tablet report, sending...
I20260927 18:40:13.888983 20523 raft_consensus.cc:348] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.889382 20523 raft_consensus.cc:729] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2b2c266f44eed8f7cb21b76ac8eba, State: Initialized, Role: FOLLOWER
I20260927 18:40:13.889820 20523 consensus_queue.cc:260] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:13.891989 20523 ts_tablet_manager.cc:1428] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:40:13.892567 20441 ts_manager.cc:194] Registered new tserver with Master: a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:13.896075 20441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:57563
I20260927 18:40:13.898904 20631 heartbeater.cc:503] Master 127.6.4.62:40247 was elected leader, sending a full tablet report...
Sep 27 18:40:13 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534413, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:40:13.904881 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.895279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.0.0.1:42808)
0927 18:40:13.895674 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:13.895717 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:13.895834 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:13.896499 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:13.896505 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:13.896920 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:13.897050 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.897058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.898041 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.898044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.898666 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:13.898676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.899366 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.899383 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:13.899465 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.900012 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:13.900028 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:13.901642 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:13.903232 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.903240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.903245 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.903452 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.903726 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:13.903729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:13.903731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:13.903810 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:13.904281 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:13.904287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:13.904401 (+ 114us) client_negotiation.cc:769] Sending connection context
0927 18:40:13.904515 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 18:40:13.904709 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":118,"threads_started":1}
I20260927 18:40:13.904902 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:13.895338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42808 (local address 127.6.4.1:44297)
0927 18:40:13.896033 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:13.896044 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:13.896066 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:40:13.896161 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:13.896166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:13.896354 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:13.896496 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:13.897158 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.897942 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.898773 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:13.899161 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:13.899258 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:13.901764 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:13.901773 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:13.901785 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:13.901807 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:13.903135 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.903551 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.903557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.903559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.903645 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:13.903885 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:13.903888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:13.903890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:13.904181 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:13.904295 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:13.904613 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 18:40:13.904763 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":82,"threads_started":1}
I20260927 18:40:13.906209 20558 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f6fa953c44a47edbb33e828bcbbf57f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:13 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:42808
I20260927 18:40:13.907162 20640 tablet_replica.cc:330] stopping tablet replica
I20260927 18:40:13.907425 20640 raft_consensus.cc:2229] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:13.907670 20640 raft_consensus.cc:2258] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:13.908603 20640 ts_tablet_manager.cc:1905] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:13.912266 20640 ts_tablet_manager.cc:1918] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:13.912348 20640 log.cc:1196] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/8f6fa953c44a47edbb33e828bcbbf57f
I20260927 18:40:13.912561 20640 ts_tablet_manager.cc:1939] T 8f6fa953c44a47edbb33e828bcbbf57f P a3a2b2c266f44eed8f7cb21b76ac8eba: Deleting consensus metadata
I20260927 18:40:13.914220 20429 catalog_manager.cc:4841] TS a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297): tablet 8f6fa953c44a47edbb33e828bcbbf57f (table test-table [id=7f817174067c4524bc62b8bf7224549e]) successfully deleted
W20260927 18:40:13.988970 20635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:13.989228 20635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:13.989276 20635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:13.993686 20635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:13.993749 20635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:13.993820 20635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:13.999032 20635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:36509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=44323
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:14.000059 20635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:14.001097 20635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:14.010381 20645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.010396 20635 server_base.cc:1034] running on GCE node
W20260927 18:40:14.010419 20646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:14.010483 20648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.011111 20635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:14.011737 20635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:14.012895 20635 hybrid_clock.cc:648] HybridClock initialized: now 1790534414012867 us; error 38 us; skew 500 ppm
Sep 27 18:40:14 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:14.015399 20635 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:14.016256 20635 webserver.cc:460] Webserver started at http://127.6.4.2:44323/ using document root <none> and password file <none>
I20260927 18:40:14.016781 20635 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:14.016856 20635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:14.020599 20635 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 18:40:14.022614 20655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.023823 20635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:14.023922 20635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "cff6f8e40df74ddd8c05069c54e86feb"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "fe797c7a653b4a652e2578b1a200d161"
server_key_iv: "14d10e1ccf51ee513d2fa48aec7e8a67"
server_key_version: "encryptionkey@0"
I20260927 18:40:14.024497 20635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:14.068030 20635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:14.070998 20635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:14.071182 20635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:14.071774 20635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:14.072597 20635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:14.072641 20635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.072695 20635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:14.072721 20635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.080973 20635 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36509
I20260927 18:40:14.081013 20768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36509 every 8 connection(s)
I20260927 18:40:14.081938 20635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:40:14.090018 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20635
I20260927 18:40:14.090380 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:37401
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:40247
--webserver_port=45741
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:40:14 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:14.094866 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.083688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:35539 (local address 127.6.4.62:40247)
0927 18:40:14.083818 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.083822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.084676 (+ 854us) server_negotiation.cc:407] Connection header received
0927 18:40:14.085916 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.085920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.085979 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.086055 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:14.087234 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.087772 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.088936 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.089170 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.089261 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.091567 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:14.091570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:14.091572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:14.091600 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:14.092952 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:14.093415 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:14.093418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:14.093420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:14.093496 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:14.093795 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:14.093799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:14.093801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:14.093995 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:14.094081 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.094563 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.094696 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:40:14.095708 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.084042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:35539)
0927 18:40:14.084532 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.084573 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.085702 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.086238 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:14.086245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:14.086588 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:14.087080 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.087095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.087903 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.087906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.088806 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.088815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.089257 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.089310 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.089473 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.090001 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:14.090017 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:14.091406 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:14.093058 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:14.093064 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:14.093079 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:14.093315 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:14.093574 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:14.093577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:14.093579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:14.093709 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:14.094092 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:14.094099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:14.094340 (+ 241us) client_negotiation.cc:769] Sending connection context
0927 18:40:14.094505 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 18:40:14.094820 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":111,"threads_started":1}
I20260927 18:40:14.096868 20769 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40247
I20260927 18:40:14.097090 20769 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:14.097563 20769 heartbeater.cc:511] Master 127.6.4.62:40247 requested a full tablet report, sending...
I20260927 18:40:14.098578 20441 ts_manager.cc:194] Registered new tserver with Master: cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509)
I20260927 18:40:14.099352 20441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:35539
W20260927 18:40:14.199183 20772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:14.199450 20772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:14.199493 20772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:14.203402 20772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:14.203459 20772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:14.203514 20772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:14.208751 20772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:37401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=45741
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:14.209762 20772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:14.210803 20772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:14.220270 20780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:14.220261 20778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:14.220270 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.220530 20772 server_base.cc:1034] running on GCE node
I20260927 18:40:14.220886 20772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:14.221469 20772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:14.222633 20772 hybrid_clock.cc:648] HybridClock initialized: now 1790534414222620 us; error 25 us; skew 500 ppm
Sep 27 18:40:14 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:14.225239 20772 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:14.226078 20772 webserver.cc:460] Webserver started at http://127.6.4.3:45741/ using document root <none> and password file <none>
I20260927 18:40:14.226605 20772 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:14.226683 20772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:14.230458 20772 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260927 18:40:14.232465 20787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.233670 20772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:14.233776 20772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "acc9732d8f1c4844ae5d6a68b911c89a"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "3c968013ecae23518356bcb348fc0062"
server_key_iv: "1b68fb2a80ab1b07b4e280fddddaa863"
server_key_version: "encryptionkey@0"
I20260927 18:40:14.234321 20772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:14.265713 20772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:14.268657 20772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:14.268838 20772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:14.269424 20772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:14.270227 20772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:14.270273 20772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.270325 20772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:14.270354 20772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.278872 20772 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37401
I20260927 18:40:14.278913 20900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37401 every 8 connection(s)
I20260927 18:40:14.279850 20772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:14.286703 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20772
Sep 27 18:40:14 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:14.293382 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.281613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:59141 (local address 127.6.4.62:40247)
0927 18:40:14.281743 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.281746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.282582 (+ 836us) server_negotiation.cc:407] Connection header received
0927 18:40:14.283851 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.283855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.283920 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.283991 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:14.285191 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.285713 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.286998 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.287297 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.287405 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.289887 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:14.289891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:14.289894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:14.289922 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:14.291309 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:14.291777 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:14.291780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:14.291781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:14.291873 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:14.292249 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:14.292252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:14.292253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:14.292466 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:14.292570 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.293104 (+ 534us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.293218 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":896}
I20260927 18:40:14.294294 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.281992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:59141)
0927 18:40:14.282434 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.282476 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.283638 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.284208 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:14.284215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:14.284570 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:14.285050 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.285064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.285828 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.285831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.286881 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.286893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.287394 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.287445 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.287647 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.288325 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:14.288341 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:14.289732 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:14.291418 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:14.291424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:14.291439 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:14.291684 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:14.291970 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:14.291973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:14.291975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:14.292170 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:14.292568 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:14.292574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:14.292856 (+ 282us) client_negotiation.cc:769] Sending connection context
0927 18:40:14.293017 (+ 161us) client_negotiation.cc:240] Negotiation successful
0927 18:40:14.293274 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":99,"threads_started":1}
I20260927 18:40:14.294862 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.287397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.6.4.62:40247)
0927 18:40:14.287647 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.287651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.287742 (+ 91us) server_negotiation.cc:407] Connection header received
0927 18:40:14.287847 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.287849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.287897 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.287979 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:14.288766 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.289337 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.290310 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.290533 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.290624 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.291761 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:14.291764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:14.291766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:14.291787 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:14.293194 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:14.293645 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:14.293648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:14.293650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:14.293727 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:14.293993 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:14.293995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:14.293996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:14.294135 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:14.294271 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.294611 (+ 340us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.294719 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"spinlock_wait_cycles":1408,"thread_start_us":68,"threads_started":1}
I20260927 18:40:14.295612 20901 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40247
I20260927 18:40:14.295840 20901 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:14.296339 20901 heartbeater.cc:511] Master 127.6.4.62:40247 requested a full tablet report, sending...
I20260927 18:40:14.297268 20442 ts_manager.cc:194] Registered new tserver with Master: acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401)
I20260927 18:40:14.297791 20442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:59141
I20260927 18:40:14.307849 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:14.311657 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.308394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.6.4.62:40247)
0927 18:40:14.308505 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.308508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.308666 (+ 158us) server_negotiation.cc:407] Connection header received
0927 18:40:14.308793 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.308796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.308885 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.308961 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:14.309116 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.309598 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.310410 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.310630 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.310700 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.310947 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:14.311379 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:14.311461 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.311515 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.311561 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:40:14.312250 20442 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:40:14.315423 20442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:40:14.322741 20558 tablet_service.cc:1467] Processing CreateTablet for tablet 3e709d0f68b9457a8eb625fd2c820b45 (DEFAULT_TABLE table=test-table [id=d9b9df0bdcbf4354b8687c54d29a8c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:14.323351 20558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e709d0f68b9457a8eb625fd2c820b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:14.326064 20523 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap starting.
I20260927 18:40:14.326915 20523 tablet_bootstrap.cc:654] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:14.327152 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.322006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36509 (local address 127.0.0.1:60964)
0927 18:40:14.322250 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.322261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.322331 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.323112 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:14.323115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:14.323133 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:14.323228 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.323234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.324481 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.324485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.325639 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.325650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.326616 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.326620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.326713 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.326752 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 18:40:14.326822 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:40:14.327019 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":121,"threads_started":1}
I20260927 18:40:14.327200 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.322419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.6.4.2:36509)
0927 18:40:14.322683 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.322693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.322714 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:40:14.322780 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.322784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.322980 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.323106 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:14.323354 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.324362 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.325760 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.326504 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.326602 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.326754 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.326974 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.327073 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
I20260927 18:40:14.328480 20523 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: No bootstrap required, opened a new log
I20260927 18:40:14.328572 20523 ts_tablet_manager.cc:1397] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:40:14.328994 20523 raft_consensus.cc:348] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.329108 20523 raft_consensus.cc:374] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:14.329142 20523 raft_consensus.cc:729] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2b2c266f44eed8f7cb21b76ac8eba, State: Initialized, Role: FOLLOWER
I20260927 18:40:14.329262 20523 consensus_queue.cc:260] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.329509 20523 ts_tablet_manager.cc:1428] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:40:14.330303 20703 tablet_service.cc:1467] Processing CreateTablet for tablet 3e709d0f68b9457a8eb625fd2c820b45 (DEFAULT_TABLE table=test-table [id=d9b9df0bdcbf4354b8687c54d29a8c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:14.330945 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.321809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37401 (local address 127.0.0.1:48528)
0927 18:40:14.321930 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.321946 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.322019 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.326781 (+ 4762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:14.326784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:14.326802 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:14.326890 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.326896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.328635 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.328638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.329479 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.329490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.330532 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.330536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.330635 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.330659 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 18:40:14.330745 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:40:14.330828 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:40:14.331323 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.322014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.6.4.3:37401)
0927 18:40:14.326293 (+ 4279us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.326304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.326330 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:40:14.326403 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.326409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.326624 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.327046 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:14.327215 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.328485 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.329637 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.330411 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.330522 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.330676 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.331078 (+ 402us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.331181 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4176,"thread_start_us":140,"threads_started":1}
I20260927 18:40:14.331372 20703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e709d0f68b9457a8eb625fd2c820b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:14.333997 20835 tablet_service.cc:1467] Processing CreateTablet for tablet 3e709d0f68b9457a8eb625fd2c820b45 (DEFAULT_TABLE table=test-table [id=d9b9df0bdcbf4354b8687c54d29a8c86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:14.334863 20835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e709d0f68b9457a8eb625fd2c820b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:14.338380 20914 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Bootstrap starting.
I20260927 18:40:14.341144 20914 tablet_bootstrap.cc:654] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:14.341598 20915 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Bootstrap starting.
I20260927 18:40:14.342000 20914 log.cc:824] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:14.343457 20915 tablet_bootstrap.cc:654] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:14.343761 20914 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: No bootstrap required, opened a new log
I20260927 18:40:14.343931 20914 ts_tablet_manager.cc:1397] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 18:40:14.344452 20915 log.cc:824] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:14.346451 20915 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: No bootstrap required, opened a new log
I20260927 18:40:14.346642 20915 ts_tablet_manager.cc:1397] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:40:14.347523 20914 raft_consensus.cc:348] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.347802 20914 raft_consensus.cc:374] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:14.347857 20914 raft_consensus.cc:729] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff6f8e40df74ddd8c05069c54e86feb, State: Initialized, Role: FOLLOWER
I20260927 18:40:14.348332 20914 consensus_queue.cc:260] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.348945 20769 heartbeater.cc:503] Master 127.6.4.62:40247 was elected leader, sending a full tablet report...
I20260927 18:40:14.349339 20914 ts_tablet_manager.cc:1428] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260927 18:40:14.350123 20915 raft_consensus.cc:348] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.350328 20915 raft_consensus.cc:374] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:14.350421 20915 raft_consensus.cc:729] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc9732d8f1c4844ae5d6a68b911c89a, State: Initialized, Role: FOLLOWER
I20260927 18:40:14.350835 20915 consensus_queue.cc:260] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.351432 20901 heartbeater.cc:503] Master 127.6.4.62:40247 was elected leader, sending a full tablet report...
I20260927 18:40:14.351680 20915 ts_tablet_manager.cc:1428] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260927 18:40:14.367628 20632 tablet.cc:2369] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:14.439113 20919 raft_consensus.cc:482] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:14.439311 20919 raft_consensus.cc:504] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.440527 20919 leader_election.cc:288] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509), a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:14.445568 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.440880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:47261 (local address 127.6.4.2:36509)
0927 18:40:14.441036 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.441042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.441271 (+ 229us) server_negotiation.cc:407] Connection header received
0927 18:40:14.441356 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.441358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.441403 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.441478 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:14.441794 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.442569 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.444356 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.444987 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.445104 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.445177 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.445351 (+ 174us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.445442 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 18:40:14.445570 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.441040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:34691 (local address 127.6.4.1:44297)
0927 18:40:14.441274 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.441278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.441292 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:40:14.441350 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.441354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.441417 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.441518 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:14.441794 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.442574 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.444356 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.445053 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.445145 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.445321 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.445362 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.445435 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260927 18:40:14.445576 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.440767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.6.4.3:34691)
0927 18:40:14.440890 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.440904 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.441006 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.441532 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:14.441534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:14.441570 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:14.441688 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.441695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.442714 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.442718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.444214 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.444226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.445150 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.445154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.445268 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.445285 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:40:14.445350 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:40:14.445404 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":122,"spinlock_wait_cycles":384}
I20260927 18:40:14.445585 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.440767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36509 (local address 127.6.4.3:47261)
0927 18:40:14.441182 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.441193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.441281 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.441495 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:14.441498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:14.441514 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:14.441698 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.441704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.442714 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.442718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.444214 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:14.444226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.445103 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.445107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:14.445213 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.445244 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 18:40:14.445319 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:40:14.445393 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":244,"spinlock_wait_cycles":14464,"thread_start_us":141,"threads_started":1}
I20260927 18:40:14.446215 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e709d0f68b9457a8eb625fd2c820b45" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" is_pre_election: true
I20260927 18:40:14.446280 20723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e709d0f68b9457a8eb625fd2c820b45" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff6f8e40df74ddd8c05069c54e86feb" is_pre_election: true
I20260927 18:40:14.446556 20572 raft_consensus.cc:2454] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 0.
I20260927 18:40:14.446570 20723 raft_consensus.cc:2454] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 0.
I20260927 18:40:14.447039 20791 leader_election.cc:302] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a2b2c266f44eed8f7cb21b76ac8eba, acc9732d8f1c4844ae5d6a68b911c89a; no voters:
I20260927 18:40:14.447503 20919 raft_consensus.cc:2781] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:14.447576 20919 raft_consensus.cc:482] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:14.447618 20919 raft_consensus.cc:3037] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:14.448691 20919 raft_consensus.cc:504] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.449016 20919 leader_election.cc:288] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 election: Requested vote from peers cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509), a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:14.449359 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e709d0f68b9457a8eb625fd2c820b45" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
I20260927 18:40:14.449364 20723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e709d0f68b9457a8eb625fd2c820b45" candidate_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cff6f8e40df74ddd8c05069c54e86feb"
I20260927 18:40:14.449484 20572 raft_consensus.cc:3037] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:14.449486 20723 raft_consensus.cc:3037] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:14.450407 20572 raft_consensus.cc:2454] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 1.
I20260927 18:40:14.450480 20723 raft_consensus.cc:2454] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc9732d8f1c4844ae5d6a68b911c89a in term 1.
I20260927 18:40:14.450664 20791 leader_election.cc:302] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a2b2c266f44eed8f7cb21b76ac8eba, acc9732d8f1c4844ae5d6a68b911c89a; no voters:
I20260927 18:40:14.450858 20919 raft_consensus.cc:2781] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:14.451045 20919 raft_consensus.cc:686] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 1 LEADER]: Becoming Leader. State: Replica: acc9732d8f1c4844ae5d6a68b911c89a, State: Running, Role: LEADER
I20260927 18:40:14.451373 20919 consensus_queue.cc:237] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.454376 20441 catalog_manager.cc:5495] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a reported cstate change: term changed from 0 to 1, leader changed from <none> to acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3). New cstate: current_term: 1 leader_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } health_report { overall_health: UNKNOWN } } }
I20260927 18:40:14.484773 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.481567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.6.4.3:37401)
0927 18:40:14.481702 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:14.481706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:14.481719 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:40:14.481795 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:14.481797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:14.481854 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:14.481941 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:14.482190 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.482686 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.483450 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:14.483657 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:14.483730 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:14.483982 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:14.484466 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:14.484552 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:14.484609 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 18:40:14.484660 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 18:40:14.490469 20723 raft_consensus.cc:1261] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Refusing update from remote peer acc9732d8f1c4844ae5d6a68b911c89a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:14.490466 20572 raft_consensus.cc:1261] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Refusing update from remote peer acc9732d8f1c4844ae5d6a68b911c89a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:14.491112 20922 consensus_queue.cc:1035] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:14.491282 20919 consensus_queue.cc:1035] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:14.496773 20924 mvcc.cc:205] Tried to move back new op lower bound from 7334028961746169856 to 7334028961595408384. Current Snapshot: MvccSnapshot[applied={T|T < 7334028961746169856}]
I20260927 18:40:14.506151 20442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:14.506309 20442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:14.508581 20442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Sending DeleteTablet for 3 replicas of tablet 3e709d0f68b9457a8eb625fd2c820b45
I20260927 18:40:14.509261 20558 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e709d0f68b9457a8eb625fd2c820b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:14 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:42808
I20260927 18:40:14.509279 20835 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e709d0f68b9457a8eb625fd2c820b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:14 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:48528
I20260927 18:40:14.509644 20933 tablet_replica.cc:330] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: stopping tablet replica
I20260927 18:40:14.509682 20934 tablet_replica.cc:330] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: stopping tablet replica
I20260927 18:40:14.509843 20933 raft_consensus.cc:2229] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:14.509865 20703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e709d0f68b9457a8eb625fd2c820b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:14 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60964
I20260927 18:40:14.510015 20934 raft_consensus.cc:2229] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:40:14.510037 20933 raft_consensus.cc:2258] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:14.510349 20934 raft_consensus.cc:2258] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:14.510454 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 20480
I20260927 18:40:14.512272 20934 ts_tablet_manager.cc:1905] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:14.514984 20934 ts_tablet_manager.cc:1918] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:14.515076 20934 log.cc:1196] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/3e709d0f68b9457a8eb625fd2c820b45
I20260927 18:40:14.515373 20934 ts_tablet_manager.cc:1939] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Deleting consensus metadata
I20260927 18:40:14.516848 20935 tablet_replica.cc:330] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: stopping tablet replica
I20260927 18:40:14.517068 20426 catalog_manager.cc:4841] TS acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401): tablet 3e709d0f68b9457a8eb625fd2c820b45 (table test-table [id=d9b9df0bdcbf4354b8687c54d29a8c86]) successfully deleted
I20260927 18:40:14.517167 20935 raft_consensus.cc:2229] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:14.517484 20935 raft_consensus.cc:2258] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 18:40:14.518389 20429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:44297 (error 108)
I20260927 18:40:14.518671 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 20635
I20260927 18:40:14.518926 20935 ts_tablet_manager.cc:1905] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:14.520547 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.520302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.0.0.1:42810)
0927 18:40:14.520411 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.520475 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:44297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260927 18:40:14.520710 20429 catalog_manager.cc:4584] TS a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e709d0f68b9457a8eb625fd2c820b45: Network error: Client connection negotiation failed: client connection to 127.6.4.1:44297: connect: Connection refused (error 111)
I20260927 18:40:14.525221 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 20772
I20260927 18:40:14.526196 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.525382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36509 (local address 127.0.0.1:60974)
0927 18:40:14.526086 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.526131 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.2:36509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633}
I20260927 18:40:14.529929 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 20409
I20260927 18:40:14.535261 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40247
--webserver_interface=127.6.4.62
--webserver_port=41535
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:14.642844 20937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:14.643106 20937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:14.643146 20937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:14.647091 20937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:14.647146 20937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:14.647164 20937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:14.647179 20937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:14.647192 20937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:14.652359 20937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=41535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:14.653368 20937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:14.654407 20937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:14.662528 20943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:14.662571 20942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.662645 20937 server_base.cc:1034] running on GCE node
W20260927 18:40:14.662571 20945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.663415 20937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:14.664479 20937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:14.665635 20937 hybrid_clock.cc:648] HybridClock initialized: now 1790534414665620 us; error 26 us; skew 500 ppm
Sep 27 18:40:14 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:14.668501 20937 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:14.669380 20937 webserver.cc:460] Webserver started at http://127.6.4.62:41535/ using document root <none> and password file <none>
I20260927 18:40:14.670013 20937 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:14.670094 20937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:14.674268 20937 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:14.676316 20952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.677543 20937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:14.677652 20937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "ce9c80012aca43d1b5c80aff8033e0cc"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "e2035b264c3c4f3bf34d6d714ee1ca46"
server_key_iv: "9145bf406a04a818d1ef9fb6d5f8154d"
server_key_version: "encryptionkey@0"
I20260927 18:40:14.678179 20937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:14.696054 20937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:14.698948 20937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:14.699119 20937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:14.705744 20937 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40247
I20260927 18:40:14.706017 21004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40247 every 8 connection(s)
I20260927 18:40:14.706802 20937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:14.710762 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 20937
I20260927 18:40:14.711249 21005 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:40:14.711249 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:44297
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:40247
--webserver_port=41925
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:14.713291 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap starting.
I20260927 18:40:14.730016 21005 log.cc:824] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:14.741236 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:14.741564 21005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc: Bootstrap complete.
I20260927 18:40:14.745405 21005 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:14.745829 21005 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9c80012aca43d1b5c80aff8033e0cc, State: Initialized, Role: FOLLOWER
I20260927 18:40:14.746323 21005 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:14.746448 21005 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:14.746516 21005 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:14.746613 21005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:40:14.748438 21005 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:14.748744 21005 leader_election.cc:302] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9c80012aca43d1b5c80aff8033e0cc; no voters:
I20260927 18:40:14.749038 21005 leader_election.cc:288] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 18:40:14.749138 21009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:40:14.749348 21009 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [term 3 LEADER]: Becoming Leader. State: Replica: ce9c80012aca43d1b5c80aff8033e0cc, State: Running, Role: LEADER
I20260927 18:40:14.749699 21009 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } }
I20260927 18:40:14.750247 21005 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:14.751096 21011 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9c80012aca43d1b5c80aff8033e0cc. Latest consensus state: current_term: 3 leader_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } } }
I20260927 18:40:14.751221 21011 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:14.751554 21018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:14.751571 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9c80012aca43d1b5c80aff8033e0cc" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40247 } } }
I20260927 18:40:14.751684 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9c80012aca43d1b5c80aff8033e0cc [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:14.756500 21018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:14.757149 21018 catalog_manager.cc:1259] Loaded cluster ID: 4759ba93e425477f867bdfd114c53797
I20260927 18:40:14.757201 21018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 18:40:14.757989 21018 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0e8860c9ad google::LogMessage::Fail() at ??:0
@ 0x7f0e8860f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0e8860c4de google::LogMessage::Flush() at ??:0
@ 0x7f0e8860da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0e92f4bfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f0e92f4cd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0e92f49579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f0e92f8d640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0e8a85e3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0e8a85ec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f0e8a8603f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0e8a84f512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0e8be936db start_thread at ??:0
@ 0x7f0e8710a61f clone at ??:0
W20260927 18:40:14.843519 21007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:14.843761 21007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:14.843808 21007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:14.847761 21007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:14.847821 21007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:14.847880 21007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:14.853116 21007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:44297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=41925
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:14.854219 21007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:14.855232 21007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:14.865231 21047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.865345 21007 server_base.cc:1034] running on GCE node
W20260927 18:40:14.865223 21046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:14.865530 21049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:14.865890 21007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:14.866551 21007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:14.867702 21007 hybrid_clock.cc:648] HybridClock initialized: now 1790534414867664 us; error 58 us; skew 500 ppm
Sep 27 18:40:14 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:14.870281 21007 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:14.871126 21007 webserver.cc:460] Webserver started at http://127.6.4.1:41925/ using document root <none> and password file <none>
I20260927 18:40:14.871659 21007 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:14.871717 21007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:14.875626 21007 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:40:14.877826 21056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:14.879063 21007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 18:40:14.879156 21007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba"
format_stamp: "Formatted at 2026-09-27 18:40:12 on dist-test-slave-bzzj"
server_key: "66354bdec6b216708df5315250ed4db3"
server_key_iv: "5ab8ce4e92b26832a18a27bb27102c66"
server_key_version: "encryptionkey@0"
I20260927 18:40:14.879700 21007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:14.906967 21007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:14.910197 21007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:14.910367 21007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:14.910972 21007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:14.912075 21063 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:40:14.917522 21007 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:40:14.917583 21007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260927 18:40:14.917646 21007 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:40:14.919390 21007 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:40:14.919448 21007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260927 18:40:14.919754 21063 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap starting.
I20260927 18:40:14.929059 21007 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:44297
I20260927 18:40:14.929155 21170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:44297 every 8 connection(s)
I20260927 18:40:14.930064 21007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:14.938788 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21007
I20260927 18:40:14.939124 21063 log.cc:824] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:14.939265 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:36509
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:40247
--webserver_port=44323
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:14.947788 21063 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:14.948211 21063 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Bootstrap complete.
I20260927 18:40:14.949529 21063 ts_tablet_manager.cc:1397] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent bootstrapping tablet: real 0.030s user 0.018s sys 0.011s
I20260927 18:40:14.952760 21063 raft_consensus.cc:348] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.953274 21063 raft_consensus.cc:729] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a2b2c266f44eed8f7cb21b76ac8eba, State: Initialized, Role: FOLLOWER
I20260927 18:40:14.953817 21063 consensus_queue.cc:260] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:14.954852 21063 ts_tablet_manager.cc:1428] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260927 18:40:14.995275 21173 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:40:14.931926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:57355)
0927 18:40:14.932597 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.932654 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:14.933719 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:14.994522 (+ 60803us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":176,"threads_started":1}
W20260927 18:40:14.995754 21171 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:40247 (0 consecutive failures): Network error: Failed to ping master at 127.6.4.62:40247: Client connection negotiation failed: client connection to 127.6.4.62:40247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:40:14.996500 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.996256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:57353)
0927 18:40:14.996372 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.996440 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:40:14.997093 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:14.996885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:36863)
0927 18:40:14.997002 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:14.997047 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 18:40:14.997236 21171 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:40:15.052819 21175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:15.053071 21175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:15.053129 21175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:15.057060 21175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:15.057111 21175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:15.057189 21175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:15.062287 21175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:36509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=44323
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:15.063256 21175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:15.064244 21175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:15.073206 21184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:15.073371 21175 server_base.cc:1034] running on GCE node
W20260927 18:40:15.073246 21181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:15.073226 21182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:15.073850 21175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:15.074437 21175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:15.075592 21175 hybrid_clock.cc:648] HybridClock initialized: now 1790534415075575 us; error 28 us; skew 500 ppm
Sep 27 18:40:15 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:15.078121 21175 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:15.078917 21175 webserver.cc:460] Webserver started at http://127.6.4.2:44323/ using document root <none> and password file <none>
I20260927 18:40:15.079429 21175 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:15.079489 21175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:15.083139 21175 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:15.085301 21191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:15.086575 21175 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:40:15.086681 21175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "cff6f8e40df74ddd8c05069c54e86feb"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "fe797c7a653b4a652e2578b1a200d161"
server_key_iv: "14d10e1ccf51ee513d2fa48aec7e8a67"
server_key_version: "encryptionkey@0"
I20260927 18:40:15.087214 21175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:15.105835 21175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:15.108755 21175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:15.108939 21175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:15.109537 21175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:15.110564 21198 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:40:15.114658 21175 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:40:15.114717 21175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260927 18:40:15.114782 21175 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:40:15.116439 21175 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:40:15.116482 21175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260927 18:40:15.116732 21198 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Bootstrap starting.
I20260927 18:40:15.122416 21198 log.cc:824] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:15.125628 21175 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:36509
I20260927 18:40:15.125735 21306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:36509 every 8 connection(s)
I20260927 18:40:15.126451 21175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:40:15.129575 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.128182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:48019)
0927 18:40:15.128627 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.128753 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":98,"threads_started":1}
W20260927 18:40:15.129962 21307 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:40247 (0 consecutive failures): Network error: Failed to ping master at 127.6.4.62:40247: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
I20260927 18:40:15.130371 21198 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:15.130537 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.130314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:41555)
0927 18:40:15.130429 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.130477 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 18:40:15.130746 21198 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Bootstrap complete.
I20260927 18:40:15.131062 21198 ts_tablet_manager.cc:1397] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20260927 18:40:15.131127 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.130922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:50553)
0927 18:40:15.131021 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.131071 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 18:40:15.131274 21307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:40:15.133828 21198 raft_consensus.cc:348] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:15.134245 21198 raft_consensus.cc:729] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff6f8e40df74ddd8c05069c54e86feb, State: Initialized, Role: FOLLOWER
I20260927 18:40:15.134660 21198 consensus_queue.cc:260] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:15.135495 21198 ts_tablet_manager.cc:1428] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260927 18:40:15.135562 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.135347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:53901)
0927 18:40:15.135444 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.135507 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:40:15.136076 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.135908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:60505)
0927 18:40:15.135987 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.136026 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:40:15.136354 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21175
I20260927 18:40:15.136698 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:37401
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:40247
--webserver_port=45741
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:15.243005 21311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:15.243250 21311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:15.243291 21311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:15.247203 21311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:15.247254 21311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:15.247320 21311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:15.252362 21311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:37401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=45741
--tserver_master_addrs=127.6.4.62:40247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:15.253371 21311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:15.254376 21311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:15.263463 21317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:15.263492 21319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:15.263463 21316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:15.263743 21311 server_base.cc:1034] running on GCE node
I20260927 18:40:15.264150 21311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:15.264767 21311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:15.265925 21311 hybrid_clock.cc:648] HybridClock initialized: now 1790534415265903 us; error 36 us; skew 500 ppm
Sep 27 18:40:15 dist-test-slave-bzzj krb5kdc[19863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:15.268393 21311 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:15.269171 21311 webserver.cc:460] Webserver started at http://127.6.4.3:45741/ using document root <none> and password file <none>
I20260927 18:40:15.269694 21311 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:15.269757 21311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:15.273429 21311 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:15.275389 21326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:15.276582 21311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:15.276684 21311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "acc9732d8f1c4844ae5d6a68b911c89a"
format_stamp: "Formatted at 2026-09-27 18:40:13 on dist-test-slave-bzzj"
server_key: "3c968013ecae23518356bcb348fc0062"
server_key_iv: "1b68fb2a80ab1b07b4e280fddddaa863"
server_key_version: "encryptionkey@0"
I20260927 18:40:15.277184 21311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:15.296065 21311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:15.298717 21311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:15.298879 21311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:15.299417 21311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:15.300230 21311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:15.300277 21311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:15.300330 21311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:15.300359 21311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:15.308429 21311 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37401
I20260927 18:40:15.308466 21439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37401 every 8 connection(s)
I20260927 18:40:15.309350 21311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:15.312191 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21311
I20260927 18:40:15.312927 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.311287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:40015)
0927 18:40:15.311697 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.312216 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
W20260927 18:40:15.313282 21440 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:40247 (0 consecutive failures): Network error: Failed to ping master at 127.6.4.62:40247: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
I20260927 18:40:15.313795 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.313611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:47649)
0927 18:40:15.313699 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.313745 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 18:40:15.314323 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.314147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:50571)
0927 18:40:15.314236 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.314278 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 18:40:15.314462 21440 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:40:15.998091 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:15.997688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:47127)
0927 18:40:15.997913 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:15.997986 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":63,"threads_started":1}
I20260927 18:40:16.137040 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.136682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:34903)
0927 18:40:16.136911 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:16.136960 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":64,"threads_started":1}
I20260927 18:40:16.315212 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.314886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:55471)
0927 18:40:16.315076 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:16.315123 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260927 18:40:16.743902 21447 raft_consensus.cc:482] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:16.744073 21447 raft_consensus.cc:504] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:16.745049 21447 leader_election.cc:288] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401), cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509)
Sep 27 18:40:16 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.2@KRBTEST.COM
Sep 27 18:40:16 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534414, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.3@KRBTEST.COM
I20260927 18:40:16.757157 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.745575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:36509 (local address 127.6.4.1:38235)
0927 18:40:16.745834 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:16.745846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:16.745937 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:16.746946 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:16.746953 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:16.747422 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:16.747906 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:16.747921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.749178 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.749181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:16.750087 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:16.750096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.750716 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.750732 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:16.750935 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:16.751550 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:16.751561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:16.752969 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:16.754995 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:16.755012 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:16.755264 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:16.755563 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:16.755568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:16.755688 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:16.756255 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:16.756261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:16.756485 (+ 224us) client_negotiation.cc:769] Sending connection context
0927 18:40:16.756636 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 18:40:16.756973 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":130,"spinlock_wait_cycles":10368,"thread_start_us":53,"threads_started":1}
I20260927 18:40:16.757171 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.745758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:38235 (local address 127.6.4.2:36509)
0927 18:40:16.746048 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:16.746059 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:16.746090 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:40:16.746268 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:16.746273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:16.746587 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:16.746828 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:16.748070 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.749047 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.750198 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.750623 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.750763 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:16.753152 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:16.753157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:16.753168 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:16.753196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:16.754887 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755362 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:16.755368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:16.755373 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:16.755479 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755770 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:16.755773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:16.755775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:16.756154 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:16.756268 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:16.756753 (+ 485us) server_negotiation.cc:299] Negotiation successful
0927 18:40:16.756982 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":69,"threads_started":1}
I20260927 18:40:16.757174 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.745357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:49935 (local address 127.6.4.3:37401)
0927 18:40:16.745685 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:16.745710 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:16.745754 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:40:16.746469 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:16.746476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:16.746765 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:16.747061 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:16.748064 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.749054 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.750190 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.750623 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.750763 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:16.753510 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:16.753514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:16.753523 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:16.753546 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:16.755024 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755423 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:16.755428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:16.755433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:16.755534 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755769 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:16.755772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:16.755774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:16.756162 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:16.756270 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:16.756756 (+ 486us) server_negotiation.cc:299] Negotiation successful
0927 18:40:16.757003 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":50,"threads_started":1}
I20260927 18:40:16.757195 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.745236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37401 (local address 127.6.4.1:49935)
0927 18:40:16.745460 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:16.745473 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:16.745566 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:16.747008 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:16.747011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:16.747419 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:16.747906 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:16.747921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.749178 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.749182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:16.750085 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:16.750094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:16.750717 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:16.750736 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:16.750937 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:16.751510 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:16.751531 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:16.753321 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:16.755124 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:16.755129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:16.755330 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:16.755616 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:16.755619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:16.755621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:16.755694 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:16.756256 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:16.756262 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:16.756484 (+ 222us) client_negotiation.cc:769] Sending connection context
0927 18:40:16.756635 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 18:40:16.757042 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":336,"spinlock_wait_cycles":32128,"thread_start_us":84,"threads_started":1}
I20260927 18:40:16.758762 21260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e709d0f68b9457a8eb625fd2c820b45" candidate_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cff6f8e40df74ddd8c05069c54e86feb" is_pre_election: true
I20260927 18:40:16.759028 21260 raft_consensus.cc:2454] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a2b2c266f44eed8f7cb21b76ac8eba in term 1.
W20260927 18:40:16.759847 21057 leader_election.cc:341] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401): Not found: Tablet not found: 3e709d0f68b9457a8eb625fd2c820b45
I20260927 18:40:16.760079 21057 leader_election.cc:302] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3a2b2c266f44eed8f7cb21b76ac8eba, cff6f8e40df74ddd8c05069c54e86feb; no voters: acc9732d8f1c4844ae5d6a68b911c89a
I20260927 18:40:16.760331 21447 raft_consensus.cc:2781] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:40:16.760399 21447 raft_consensus.cc:482] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:16.760448 21447 raft_consensus.cc:3037] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:40:16.762414 21447 raft_consensus.cc:504] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:16.762709 21447 leader_election.cc:288] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [CANDIDATE]: Term 2 election: Requested vote from peers acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401), cff6f8e40df74ddd8c05069c54e86feb (127.6.4.2:36509)
I20260927 18:40:16.763062 21260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e709d0f68b9457a8eb625fd2c820b45" candidate_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cff6f8e40df74ddd8c05069c54e86feb"
I20260927 18:40:16.763191 21260 raft_consensus.cc:3037] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 1 FOLLOWER]: Advancing to term 2
W20260927 18:40:16.763306 21057 leader_election.cc:341] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401): Not found: Tablet not found: 3e709d0f68b9457a8eb625fd2c820b45
I20260927 18:40:16.765102 21260 raft_consensus.cc:2454] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a2b2c266f44eed8f7cb21b76ac8eba in term 2.
I20260927 18:40:16.765368 21057 leader_election.cc:302] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3a2b2c266f44eed8f7cb21b76ac8eba, cff6f8e40df74ddd8c05069c54e86feb; no voters: acc9732d8f1c4844ae5d6a68b911c89a
I20260927 18:40:16.765513 21447 raft_consensus.cc:2781] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:40:16.765659 21447 raft_consensus.cc:686] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [term 2 LEADER]: Becoming Leader. State: Replica: a3a2b2c266f44eed8f7cb21b76ac8eba, State: Running, Role: LEADER
I20260927 18:40:16.765944 21447 consensus_queue.cc:237] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:16.766188 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:16.766002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:50187)
0927 18:40:16.766094 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:16.766139 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:40:17.138005 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.137656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:36419)
0927 18:40:17.137862 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.137923 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
W20260927 18:40:17.173856 21057 consensus_peers.cc:487] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba -> Peer acc9732d8f1c4844ae5d6a68b911c89a (127.6.4.3:37401): Couldn't send request to peer acc9732d8f1c4844ae5d6a68b911c89a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e709d0f68b9457a8eb625fd2c820b45. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:40:17.299142 21260 raft_consensus.cc:1261] T 3e709d0f68b9457a8eb625fd2c820b45 P cff6f8e40df74ddd8c05069c54e86feb [term 2 FOLLOWER]: Refusing update from remote peer a3a2b2c266f44eed8f7cb21b76ac8eba: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260927 18:40:17.299803 21447 consensus_queue.cc:1035] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba [LEADER]: Connected to new peer: Peer: permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260927 18:40:17.301555 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.301308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:39247)
0927 18:40:17.301419 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.301475 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:40:17.304013 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.303701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:50141)
0927 18:40:17.303874 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.303929 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":49,"threads_started":1}
I20260927 18:40:17.304067 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.303860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:41541)
0927 18:40:17.303958 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.304011 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:40:17.316064 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.315753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:55163)
0927 18:40:17.315934 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.315979 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260927 18:40:17.707563 21462 ts_tablet_manager.cc:927] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Initiating tablet copy from peer a3a2b2c266f44eed8f7cb21b76ac8eba (127.6.4.1:44297)
I20260927 18:40:17.708139 21462 tablet_copy_client.cc:316] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: tablet copy: Beginning tablet copy session from remote peer at address 127.6.4.1:44297
Sep 27 18:40:17 dist-test-slave-bzzj krb5kdc[19863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534415, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:40:17.721264 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.708585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44297 (local address 127.6.4.3:48997)
0927 18:40:17.708695 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.708715 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:17.708833 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:17.709388 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:17.709392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:17.709749 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:17.709860 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:17.709866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:17.710794 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:17.710796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:17.711420 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:17.711427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:17.711971 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:17.711987 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:17.712065 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:17.716812 (+ 4747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:17.716825 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:17.718118 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:17.719849 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:17.719857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:17.719862 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:17.720065 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:17.720358 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:17.720360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:17.720362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:17.720432 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:17.720798 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:17.720803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:17.720902 (+ 99us) client_negotiation.cc:769] Sending connection context
0927 18:40:17.721009 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 18:40:17.721136 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260927 18:40:17.721313 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.708709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:48997 (local address 127.6.4.1:44297)
0927 18:40:17.709007 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:17.709019 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:17.709041 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:40:17.709085 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:17.709090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:17.709264 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:17.709379 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:17.709978 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:17.710705 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:17.711522 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:17.711872 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:17.711982 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:17.718243 (+ 6261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:17.718252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:17.718264 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:17.718287 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:17.719760 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:17.720208 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:17.720213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:17.720214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:17.720291 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:17.720491 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:17.720493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:17.720494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:17.720720 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:17.720819 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:17.721074 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 18:40:17.721191 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260927 18:40:17.722282 21145 tablet_copy_service.cc:140] P a3a2b2c266f44eed8f7cb21b76ac8eba: Received BeginTabletCopySession request for tablet 3e709d0f68b9457a8eb625fd2c820b45 from peer acc9732d8f1c4844ae5d6a68b911c89a ({username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:48997)
I20260927 18:40:17.722424 21145 tablet_copy_service.cc:161] P a3a2b2c266f44eed8f7cb21b76ac8eba: Beginning new tablet copy session on tablet 3e709d0f68b9457a8eb625fd2c820b45 from peer acc9732d8f1c4844ae5d6a68b911c89a at {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:48997: session id = acc9732d8f1c4844ae5d6a68b911c89a-3e709d0f68b9457a8eb625fd2c820b45
I20260927 18:40:17.723477 21145 tablet_copy_source_session.cc:205] T 3e709d0f68b9457a8eb625fd2c820b45 P a3a2b2c266f44eed8f7cb21b76ac8eba: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:40:17.727312 21462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e709d0f68b9457a8eb625fd2c820b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:17.733244 21462 tablet_copy_client.cc:791] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: tablet copy: Starting download of 0 data blocks...
I20260927 18:40:17.733412 21462 tablet_copy_client.cc:655] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: tablet copy: Starting download of 1 WAL segments...
I20260927 18:40:17.734865 21462 tablet_copy_client.cc:531] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:40:17.736394 21462 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Bootstrap starting.
I20260927 18:40:17.741262 21462 log.cc:824] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:17.747160 21462 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:17.747426 21462 tablet_bootstrap.cc:492] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Bootstrap complete.
I20260927 18:40:17.747695 21462 ts_tablet_manager.cc:1397] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260927 18:40:17.749770 21462 raft_consensus.cc:348] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:17.749975 21462 raft_consensus.cc:729] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc9732d8f1c4844ae5d6a68b911c89a, State: Initialized, Role: FOLLOWER
I20260927 18:40:17.750336 21462 consensus_queue.cc:260] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acc9732d8f1c4844ae5d6a68b911c89a" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37401 } } peers { permanent_uuid: "cff6f8e40df74ddd8c05069c54e86feb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 36509 } } peers { permanent_uuid: "a3a2b2c266f44eed8f7cb21b76ac8eba" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44297 } }
I20260927 18:40:17.751142 21462 ts_tablet_manager.cc:1428] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260927 18:40:17.751400 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.751128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:46605)
0927 18:40:17.751270 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.751322 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 18:40:17.751711 21145 tablet_copy_service.cc:342] P a3a2b2c266f44eed8f7cb21b76ac8eba: Request end of tablet copy session acc9732d8f1c4844ae5d6a68b911c89a-3e709d0f68b9457a8eb625fd2c820b45 received from {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:48997
I20260927 18:40:17.751821 21145 tablet_copy_service.cc:433] P a3a2b2c266f44eed8f7cb21b76ac8eba: ending tablet copy session acc9732d8f1c4844ae5d6a68b911c89a-3e709d0f68b9457a8eb625fd2c820b45 on tablet 3e709d0f68b9457a8eb625fd2c820b45 with peer acc9732d8f1c4844ae5d6a68b911c89a
I20260927 18:40:17.751971 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:17.751753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:39837)
0927 18:40:17.751861 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:17.751910 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:40:18.215857 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:18.215504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:39221)
0927 18:40:18.215727 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:18.215770 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":61,"threads_started":1}
I20260927 18:40:18.215916 21394 raft_consensus.cc:1203] T 3e709d0f68b9457a8eb625fd2c820b45 P acc9732d8f1c4844ae5d6a68b911c89a [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260927 18:40:18.216771 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:18.216572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:50895)
0927 18:40:18.216666 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:18.216714 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:40:18.305091 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:18.304666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:56467)
0927 18:40:18.304941 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:18.304996 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":63,"threads_started":1}
I20260927 18:40:19.216867 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:19.216485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:48177)
0927 18:40:19.216728 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:19.216777 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260927 18:40:19.217787 21472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:19.217404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:35093)
0927 18:40:19.217658 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:19.217709 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":60,"threads_started":1}
I20260927 18:40:19.306142 21473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:19.305739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:48713)
0927 18:40:19.305992 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:19.306047 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260927 18:40:20.218122 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:20.217708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:58471)
0927 18:40:20.217975 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:20.218029 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
I20260927 18:40:20.218823 21475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:20.218448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:47347)
0927 18:40:20.218662 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:20.218722 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
I20260927 18:40:20.218843 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:20.218632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:59325)
0927 18:40:20.218742 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:20.218786 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 18:40:20.307291 21476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:20.306911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:50297)
0927 18:40:20.307146 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:20.307196 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
I20260927 18:40:21.219892 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:21.219520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:58925)
0927 18:40:21.219751 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:21.219800 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":65,"threads_started":1}
I20260927 18:40:21.220089 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:21.219639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:47825)
0927 18:40:21.219881 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:21.219931 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":64,"threads_started":1}
I20260927 18:40:21.308337 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:21.307938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:33927)
0927 18:40:21.308191 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:21.308244 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
I20260927 18:40:22.221004 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:22.220605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:45075)
0927 18:40:22.220854 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:22.220908 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260927 18:40:22.221141 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:22.220789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:53221)
0927 18:40:22.221000 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:22.221049 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":61,"threads_started":1}
I20260927 18:40:22.314572 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:22.309063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:38565)
0927 18:40:22.314401 (+ 5338us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:22.314465 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5254,"thread_start_us":85,"threads_started":1}
I20260927 18:40:23.222110 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:23.221704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:55433)
0927 18:40:23.221957 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:23.222008 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260927 18:40:23.222175 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:23.221795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:45929)
0927 18:40:23.222036 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:23.222088 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":68,"threads_started":1}
I20260927 18:40:23.315704 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:23.315323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:38141)
0927 18:40:23.315555 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:23.315619 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":66,"threads_started":1}
I20260927 18:40:24.223174 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:24.222771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:41955)
0927 18:40:24.223030 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:24.223083 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260927 18:40:24.223224 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:24.222771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:33453)
0927 18:40:24.223096 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:24.223143 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":88,"threads_started":1}
I20260927 18:40:24.316732 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:24.316357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:43855)
0927 18:40:24.316591 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:24.316645 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
I20260927 18:40:25.224299 21489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:25.223855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:43027)
0927 18:40:25.224126 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:25.224189 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260927 18:40:25.224330 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:25.223953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:36215)
0927 18:40:25.224201 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:25.224252 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":65,"threads_started":1}
I20260927 18:40:25.318138 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:25.317631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:36645)
0927 18:40:25.317971 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:25.318039 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":91,"threads_started":1}
I20260927 18:40:26.225520 21492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:26.225084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:48611)
0927 18:40:26.225356 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:26.225417 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
I20260927 18:40:26.225543 21493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:26.225133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:43807)
0927 18:40:26.225403 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:26.225456 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260927 18:40:26.319391 21494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:26.318941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:33169)
0927 18:40:26.319225 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:26.319293 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260927 18:40:27.226723 21495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:27.226306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:43153)
0927 18:40:27.226570 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:27.226626 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260927 18:40:27.226732 21496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:27.226307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:42967)
0927 18:40:27.226591 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:27.226645 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":86,"threads_started":1}
I20260927 18:40:27.320607 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:27.320154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:50739)
0927 18:40:27.320425 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:27.320496 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260927 18:40:28.227861 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:28.227451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:49349)
0927 18:40:28.227708 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:28.227763 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260927 18:40:28.227873 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:28.227447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:36461)
0927 18:40:28.227735 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:28.227788 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
I20260927 18:40:28.321772 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:28.321303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:50903)
0927 18:40:28.321591 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:28.321671 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260927 18:40:29.228894 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:29.228495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:43797)
0927 18:40:29.228745 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:29.228801 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260927 18:40:29.228916 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:29.228495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:56425)
0927 18:40:29.228769 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:29.228825 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
I20260927 18:40:29.322819 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:29.322447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:53903)
0927 18:40:29.322673 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:29.322725 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
I20260927 18:40:30.230191 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:30.229773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.1:45375)
0927 18:40:30.230023 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:30.230090 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260927 18:40:30.230250 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:30.229739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.3:52357)
0927 18:40:30.230106 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:30.230163 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":106,"threads_started":1}
I20260927 18:40:30.320928 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21007
I20260927 18:40:30.324208 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:30.323783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40247 (local address 127.6.4.2:50849)
0927 18:40:30.324025 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:30.324085 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:40247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260927 18:40:30.326452 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21175
I20260927 18:40:30.331503 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21311
2026-09-27T18:40:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18939 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:40:30 dist-test-slave-bzzj krb5kdc[21509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:40:30 dist-test-slave-bzzj krb5kdc[21509](info): set up 2 sockets
Sep 27 18:40:30 dist-test-slave-bzzj krb5kdc[21509](info): commencing operation
krb5kdc: starting...
W20260927 18:40:31.284024 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:40:31 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:40:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:40:31.399293 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44179
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:44179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:31.508999 21522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:31.509274 21522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:31.509312 21522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:31.513377 21522 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:40:31.513434 21522 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:40:31.513449 21522 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:40:31.513461 21522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:31.513478 21522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:31.513496 21522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:31.513517 21522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:31.513536 21522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:31.518834 21522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:44179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:31.519856 21522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:31.520888 21522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:31.529323 21528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:31.529317 21527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:31.529368 21522 server_base.cc:1034] running on GCE node
W20260927 18:40:31.529402 21530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:31.530071 21522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:31.531148 21522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:31.532313 21522 hybrid_clock.cc:648] HybridClock initialized: now 1790534431532294 us; error 29 us; skew 500 ppm
Sep 27 18:40:31 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:31.540908 21522 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:31.541854 21522 webserver.cc:460] Webserver started at http://127.6.4.62:46287/ using document root <none> and password file <none>
I20260927 18:40:31.542418 21522 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:31.542484 21522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:31.542685 21522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:31.544855 21522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "96bf12e6b1f147b0b44852bad50834fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "77ab79e6172c0f478fb3d979bd058f29"
server_key_iv: "eccda889c4378631204a92e72debd868"
server_key_version: "encryptionkey@0"
I20260927 18:40:31.545379 21522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "96bf12e6b1f147b0b44852bad50834fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "77ab79e6172c0f478fb3d979bd058f29"
server_key_iv: "eccda889c4378631204a92e72debd868"
server_key_version: "encryptionkey@0"
I20260927 18:40:31.549090 21522 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:31.551450 21537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:31.552651 21522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:31.552768 21522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "96bf12e6b1f147b0b44852bad50834fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "77ab79e6172c0f478fb3d979bd058f29"
server_key_iv: "eccda889c4378631204a92e72debd868"
server_key_version: "encryptionkey@0"
I20260927 18:40:31.552898 21522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:31.567245 21522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:31.570096 21522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:31.570261 21522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:31.576965 21522 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:44179
I20260927 18:40:31.577221 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:44179 every 8 connection(s)
I20260927 18:40:31.577916 21522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:31.580662 21590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:31.585223 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21522
I20260927 18:40:31.585319 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:40:31.585587 6160 external_mini_cluster.cc:1428] Setting key 5d8153cc3d06256da599f353972fa503
I20260927 18:40:31.586227 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap starting.
I20260927 18:40:31.588990 21590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:31.589876 21590 log.cc:824] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:31.591895 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: No bootstrap required, opened a new log
Sep 27 18:40:31 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:31.595340 21590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:31.595626 21590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:31.595690 21590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96bf12e6b1f147b0b44852bad50834fe, State: Initialized, Role: FOLLOWER
I20260927 18:40:31.596196 21590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:31.596343 21590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:31.596413 21590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:31.596525 21590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:31.597376 21590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:31.597702 21590 leader_election.cc:302] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96bf12e6b1f147b0b44852bad50834fe; no voters:
I20260927 18:40:31.597982 21590 leader_election.cc:288] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:40:31.598070 21595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:31.598235 21595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 LEADER]: Becoming Leader. State: Replica: 96bf12e6b1f147b0b44852bad50834fe, State: Running, Role: LEADER
I20260927 18:40:31.598587 21595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:31.599313 21590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:31.599659 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:31.586972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48988 (local address 127.6.4.62:44179)
0927 18:40:31.587399 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:31.587418 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:31.587463 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:40:31.588314 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:31.588336 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:31.588817 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:31.589218 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:31.589869 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:31.591229 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:31.592120 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:31.592599 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:31.592743 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:31.595034 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:31.595038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:31.595052 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:31.595077 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:31.596870 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:31.597255 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:31.597261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:31.597266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:31.597387 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:31.597664 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:31.597667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:31.597669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:31.598108 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:31.598283 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:31.598568 (+ 285us) server_negotiation.cc:299] Negotiation successful
0927 18:40:31.598863 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260927 18:40:31.599938 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96bf12e6b1f147b0b44852bad50834fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } } }
I20260927 18:40:31.600263 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:31.600507 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bf12e6b1f147b0b44852bad50834fe. Latest consensus state: current_term: 1 leader_uuid: "96bf12e6b1f147b0b44852bad50834fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } } }
I20260927 18:40:31.600646 21604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:31.600649 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:31.603821 21604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:31.610008 21604 catalog_manager.cc:1347] Generated new cluster ID: 236d34faf7254f3aa56b28da5d6341c1
I20260927 18:40:31.610071 21604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:31.619967 21604 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:40:31.620949 21604 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:31.625393 21604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Generated new TSK 0
I20260927 18:40:31.626068 21604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:40:31.658825 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:44179
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:31.771142 21616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:31.771420 21616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:31.771466 21616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:31.775578 21616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:31.775638 21616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:31.775695 21616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:31.781041 21616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:31.782068 21616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:31.783053 21616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:31.792543 21624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:31.792665 21616 server_base.cc:1034] running on GCE node
W20260927 18:40:31.792471 21622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:31.792484 21621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:31.793185 21616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:31.793810 21616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:31.794963 21616 hybrid_clock.cc:648] HybridClock initialized: now 1790534431794939 us; error 46 us; skew 500 ppm
Sep 27 18:40:31 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:31.797565 21616 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:31.798372 21616 webserver.cc:460] Webserver started at http://127.6.4.1:42227/ using document root <none> and password file <none>
I20260927 18:40:31.798884 21616 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:31.798941 21616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:31.799150 21616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:31.801344 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "fb5df9393f1a4af6b3f4913dcd8080fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "bd7a59c830bfcad54eecb62268040d9c"
server_key_iv: "d50d31202211c01cd0706bebf1122b75"
server_key_version: "encryptionkey@0"
I20260927 18:40:31.801913 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "fb5df9393f1a4af6b3f4913dcd8080fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "bd7a59c830bfcad54eecb62268040d9c"
server_key_iv: "d50d31202211c01cd0706bebf1122b75"
server_key_version: "encryptionkey@0"
I20260927 18:40:31.805523 21616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:31.807893 21631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:31.809126 21616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:31.809219 21616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "fb5df9393f1a4af6b3f4913dcd8080fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "bd7a59c830bfcad54eecb62268040d9c"
server_key_iv: "d50d31202211c01cd0706bebf1122b75"
server_key_version: "encryptionkey@0"
I20260927 18:40:31.809321 21616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:31.819828 21616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:31.822980 21616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:31.823156 21616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:31.823717 21616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:31.824558 21616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:31.824612 21616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:31.824656 21616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:31.824684 21616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:31.832787 21616 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:40859
I20260927 18:40:31.832818 21744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:40859 every 8 connection(s)
I20260927 18:40:31.833639 21616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:31.834748 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21616
I20260927 18:40:31.834854 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:40:31.835191 6160 external_mini_cluster.cc:1428] Setting key 975073e21a95e0ff64c69c08422e27b6
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
Sep 27 18:40:31 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:40:31.847469 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:31.835576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:39815 (local address 127.6.4.62:44179)
0927 18:40:31.835732 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:31.835737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:31.836757 (+ 1020us) server_negotiation.cc:407] Connection header received
0927 18:40:31.838573 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:31.838576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:31.838620 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:31.838683 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:31.840046 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:31.840618 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:31.841728 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:31.841957 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:31.842045 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:31.844332 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:31.844335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:31.844337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:31.844362 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:31.845504 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:31.845969 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:31.845971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:31.845972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:31.846068 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:31.846327 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:31.846330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:31.846332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:31.846548 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:31.846642 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:31.847170 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 18:40:31.847298 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:40:31.848536 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:31.836021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:39815)
0927 18:40:31.836606 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:31.836668 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:31.838328 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:31.838892 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:31.838902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:31.839396 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:31.839902 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:31.839915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:31.840733 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:31.840736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:31.841615 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:31.841624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:31.842067 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:31.842122 (+ 55us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:31.842277 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:31.842806 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:31.842822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:31.844181 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:31.845600 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:31.845605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:31.845617 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:31.845891 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:31.846133 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:31.846135 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:31.846137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:31.846255 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:31.846674 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:31.846680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:31.846917 (+ 237us) client_negotiation.cc:769] Sending connection context
0927 18:40:31.847102 (+ 185us) client_negotiation.cc:240] Negotiation successful
0927 18:40:31.847444 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":127,"threads_started":1}
I20260927 18:40:31.849758 21745 heartbeater.cc:347] Connected to a master server at 127.6.4.62:44179
I20260927 18:40:31.850031 21745 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:31.850658 21745 heartbeater.cc:511] Master 127.6.4.62:44179 requested a full tablet report, sending...
I20260927 18:40:31.852262 21554 ts_manager.cc:194] Registered new tserver with Master: fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859)
I20260927 18:40:31.853389 21554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:39815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:40:31.862313 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:44179
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:31.970649 21750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:31.970920 21750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:31.970978 21750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:31.974949 21750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:31.975004 21750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:31.975091 21750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:31.980206 21750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:31.981240 21750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:31.982265 21750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:31.991963 21758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:31.992018 21750 server_base.cc:1034] running on GCE node
W20260927 18:40:31.991899 21756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:31.991902 21755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:31.992570 21750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:31.993211 21750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:31.994366 21750 hybrid_clock.cc:648] HybridClock initialized: now 1790534431994346 us; error 36 us; skew 500 ppm
Sep 27 18:40:31 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:31.997032 21750 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:31.997896 21750 webserver.cc:460] Webserver started at http://127.6.4.2:35675/ using document root <none> and password file <none>
I20260927 18:40:31.998438 21750 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:31.998504 21750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:31.998742 21750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:32.000942 21750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "aba9ede6f4cfb54913a4ee0d79daa226"
server_key_iv: "f1db9507cb733f676432544839ece482"
server_key_version: "encryptionkey@0"
I20260927 18:40:32.001485 21750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "aba9ede6f4cfb54913a4ee0d79daa226"
server_key_iv: "f1db9507cb733f676432544839ece482"
server_key_version: "encryptionkey@0"
I20260927 18:40:32.005411 21750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260927 18:40:32.007794 21765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:32.009050 21750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:32.009159 21750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "aba9ede6f4cfb54913a4ee0d79daa226"
server_key_iv: "f1db9507cb733f676432544839ece482"
server_key_version: "encryptionkey@0"
I20260927 18:40:32.009258 21750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:32.034803 21750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:32.037909 21750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:32.038100 21750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:32.038722 21750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:32.039568 21750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:32.039615 21750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:32.039671 21750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:32.039695 21750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:32.048074 21750 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:41185
I20260927 18:40:32.048208 21878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:41185 every 8 connection(s)
I20260927 18:40:32.049031 21750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:40:32 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:32.058501 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21750
I20260927 18:40:32.058626 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:40:32.058912 6160 external_mini_cluster.cc:1428] Setting key 8183c7ccdee59f63398ec42753f0880c
I20260927 18:40:32.062019 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.050779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:60527 (local address 127.6.4.62:44179)
0927 18:40:32.050971 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.050976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.052011 (+ 1035us) server_negotiation.cc:407] Connection header received
0927 18:40:32.052895 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.052899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.052954 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.053029 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:32.054223 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.054758 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.055893 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.056154 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.056261 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.058641 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:32.058645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:32.058647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:32.058676 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:32.060047 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:32.060574 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:32.060578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:32.060580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:32.060673 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:32.060968 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:32.060971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:32.060972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:32.061181 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:32.061271 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.061730 (+ 459us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.061873 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 18:40:32.063062 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.051208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:60527)
0927 18:40:32.051843 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.051886 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.052701 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.053208 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.053215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.053562 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:32.054082 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.054096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.054870 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.054873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.055783 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.055793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.056248 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.056302 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.056466 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.057011 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:32.057027 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:32.058463 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:32.060173 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:32.060179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:32.060194 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:32.060467 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:32.060760 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:32.060763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:32.060764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:32.060890 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:32.061262 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:32.061267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:32.061507 (+ 240us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.061668 (+ 161us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.062021 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":111,"threads_started":1}
I20260927 18:40:32.064248 21879 heartbeater.cc:347] Connected to a master server at 127.6.4.62:44179
I20260927 18:40:32.064527 21879 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:32.065125 21879 heartbeater.cc:511] Master 127.6.4.62:44179 requested a full tablet report, sending...
I20260927 18:40:32.066308 21554 ts_manager.cc:194] Registered new tserver with Master: 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185)
I20260927 18:40:32.067070 21554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:60527
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:40:32.085278 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:44179
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:32.195035 21884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:32.195297 21884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:32.195355 21884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:32.199347 21884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:32.199404 21884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:32.199484 21884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:32.204830 21884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:32.205891 21884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:32.206895 21884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:32.216554 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:32.216575 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:32.216563 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:32.216655 21884 server_base.cc:1034] running on GCE node
I20260927 18:40:32.217306 21884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:32.217900 21884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:32.219054 21884 hybrid_clock.cc:648] HybridClock initialized: now 1790534432219035 us; error 32 us; skew 500 ppm
Sep 27 18:40:32 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:32.221714 21884 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:32.222592 21884 webserver.cc:460] Webserver started at http://127.6.4.3:39723/ using document root <none> and password file <none>
I20260927 18:40:32.223140 21884 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:32.223203 21884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:32.223385 21884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:40:32.225423 21884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "3507259df9be4a908c33406f7a8642cc"
format_stamp: "Formatted at 2026-09-27 18:40:32 on dist-test-slave-bzzj"
server_key: "68830f3fa28a00eacb89a67227e52a31"
server_key_iv: "c9076fb3d4d1cc5ef93011b02eeb0572"
server_key_version: "encryptionkey@0"
I20260927 18:40:32.225955 21884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "3507259df9be4a908c33406f7a8642cc"
format_stamp: "Formatted at 2026-09-27 18:40:32 on dist-test-slave-bzzj"
server_key: "68830f3fa28a00eacb89a67227e52a31"
server_key_iv: "c9076fb3d4d1cc5ef93011b02eeb0572"
server_key_version: "encryptionkey@0"
I20260927 18:40:32.229635 21884 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:40:32.232051 21899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:32.233280 21884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:40:32.233393 21884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "3507259df9be4a908c33406f7a8642cc"
format_stamp: "Formatted at 2026-09-27 18:40:32 on dist-test-slave-bzzj"
server_key: "68830f3fa28a00eacb89a67227e52a31"
server_key_iv: "c9076fb3d4d1cc5ef93011b02eeb0572"
server_key_version: "encryptionkey@0"
I20260927 18:40:32.233492 21884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:32.249820 21884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:32.253021 21884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:32.253197 21884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:32.253791 21884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:32.254621 21884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:32.254667 21884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:32.254720 21884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:32.254748 21884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:32.262871 21884 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:36339
I20260927 18:40:32.262895 22012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:36339 every 8 connection(s)
I20260927 18:40:32.263763 21884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:32.270828 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 21884
I20260927 18:40:32.270958 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:40:32.271229 6160 external_mini_cluster.cc:1428] Setting key 42a9251588a02ac0e1a38c580dcf001b
Sep 27 18:40:32 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:32.276779 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.265563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:43499 (local address 127.6.4.62:44179)
0927 18:40:32.265694 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.265698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.266551 (+ 853us) server_negotiation.cc:407] Connection header received
0927 18:40:32.267841 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.267844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.267904 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.267980 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:32.269163 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.269705 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.270893 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.271171 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.271262 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.273822 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:32.273826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:32.273828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:32.273862 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:32.275131 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:32.275530 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:32.275533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:32.275534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:32.275602 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:32.275841 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:32.275843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:32.275844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:32.276003 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:32.276071 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.276560 (+ 489us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.276661 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:40:32.277619 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.265942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:43499)
0927 18:40:32.266397 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.266441 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.267634 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.268189 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.268197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.268548 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:32.269025 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.269039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.269817 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.269820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.270765 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.270775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.271268 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.271327 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.271517 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.272063 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:32.272080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:32.273690 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:32.275214 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:32.275218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:32.275228 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:32.275459 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:32.275661 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:32.275663 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:32.275664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:32.275783 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:32.276071 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:32.276075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:32.276336 (+ 261us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.276493 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.276743 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":101,"threads_started":1}
I20260927 18:40:32.278657 22013 heartbeater.cc:347] Connected to a master server at 127.6.4.62:44179
I20260927 18:40:32.278867 22013 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:32.279309 22013 heartbeater.cc:511] Master 127.6.4.62:44179 requested a full tablet report, sending...
I20260927 18:40:32.280264 21554 ts_manager.cc:194] Registered new tserver with Master: 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
I20260927 18:40:32.280798 21554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:43499
I20260927 18:40:32.284910 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:32.293548 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.286745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.6.4.62:44179)
0927 18:40:32.286858 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.286863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.287023 (+ 160us) server_negotiation.cc:407] Connection header received
0927 18:40:32.287149 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.287152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.287189 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.287247 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:32.287886 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.288426 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.289234 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.289443 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.289534 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.290758 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:32.290762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:32.290763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:32.290785 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:32.292014 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:32.292409 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:32.292413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:32.292414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:32.292494 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:32.292732 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:32.292735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:32.292736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:32.292943 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:32.293063 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.293281 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.293388 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:40:32.296355 21554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:40:32.299445 21554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:40:32.313576 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.308794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:40859 (local address 127.0.0.1:45926)
0927 18:40:32.309020 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.309033 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.309105 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.309749 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.309755 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.309778 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:32.309923 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.309930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.310964 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.310968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.312059 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.312068 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.313056 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.313059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.313163 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.313202 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.313308 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.313423 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260927 18:40:32.313690 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.308975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45926 (local address 127.6.4.1:40859)
0927 18:40:32.309321 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.309335 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.309358 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:40:32.309429 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.309434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.309597 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.309734 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:32.310030 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.310834 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.312214 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.312944 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.313052 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.313200 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.313400 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.313507 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":82,"threads_started":1}
I20260927 18:40:32.314440 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.308148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:36339 (local address 127.0.0.1:37380)
0927 18:40:32.308788 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.308801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.308884 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.310043 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.310045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.310063 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:32.310137 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.310142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.311509 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.311512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.312814 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.312821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.314046 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.314049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.314147 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.314171 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.314237 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.314307 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":28,"thread_start_us":97,"threads_started":1}
I20260927 18:40:32.314482 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.309214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.6.4.3:36339)
0927 18:40:32.309626 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.309637 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.309658 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:40:32.309727 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.309732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.309906 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.310039 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:32.310239 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.311356 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.312927 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.313752 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.313856 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.313986 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.314267 (+ 281us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.314358 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":86,"threads_started":1}
I20260927 18:40:32.315919 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.308043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41185 (local address 127.0.0.1:42732)
0927 18:40:32.308511 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.308557 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.308676 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.309867 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.309870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.309886 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:32.310030 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.310037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.311470 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.311473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.312782 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.312793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.314089 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.314092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.315628 (+ 1536us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.315648 (+ 20us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.315718 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.315790 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":91,"thread_start_us":112,"threads_started":1}
I20260927 18:40:32.316208 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.308209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.6.4.2:41185)
0927 18:40:32.308832 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.308843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.308866 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:40:32.309304 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.309311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.309532 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.309666 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:32.310153 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.311310 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.312907 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.313966 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.314068 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.314202 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.315764 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.315858 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":109,"threads_started":1}
I20260927 18:40:32.317425 21947 tablet_service.cc:1467] Processing CreateTablet for tablet 4981516f0939451baf16bfb401c1815d (DEFAULT_TABLE table=test-table [id=914477ea494244b78b71734caf5119c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:32.317287 21679 tablet_service.cc:1467] Processing CreateTablet for tablet 4981516f0939451baf16bfb401c1815d (DEFAULT_TABLE table=test-table [id=914477ea494244b78b71734caf5119c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:32.318441 21947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4981516f0939451baf16bfb401c1815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:32.318445 21679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4981516f0939451baf16bfb401c1815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:32.318322 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 4981516f0939451baf16bfb401c1815d (DEFAULT_TABLE table=test-table [id=914477ea494244b78b71734caf5119c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:32.319180 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4981516f0939451baf16bfb401c1815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:32.324949 22030 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Bootstrap starting.
I20260927 18:40:32.326170 22031 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap starting.
I20260927 18:40:32.326584 22032 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Bootstrap starting.
I20260927 18:40:32.327440 22030 tablet_bootstrap.cc:654] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:32.328446 22032 tablet_bootstrap.cc:654] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:32.328529 22030 log.cc:824] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:32.329617 22032 log.cc:824] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:32.330155 22031 tablet_bootstrap.cc:654] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:32.330365 22030 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: No bootstrap required, opened a new log
I20260927 18:40:32.330555 22030 ts_tablet_manager.cc:1397] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 18:40:32.331255 22031 log.cc:824] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:32.331597 22032 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: No bootstrap required, opened a new log
I20260927 18:40:32.331799 22032 ts_tablet_manager.cc:1397] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:40:32.333298 22031 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: No bootstrap required, opened a new log
I20260927 18:40:32.333491 22031 ts_tablet_manager.cc:1397] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260927 18:40:32.334342 22030 raft_consensus.cc:348] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.334733 22030 raft_consensus.cc:374] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:32.334798 22030 raft_consensus.cc:729] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd862bbed3a46ad836a7dc8bb0aad6a, State: Initialized, Role: FOLLOWER
I20260927 18:40:32.334682 22032 raft_consensus.cc:348] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.334939 22032 raft_consensus.cc:374] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:32.335005 22032 raft_consensus.cc:729] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3507259df9be4a908c33406f7a8642cc, State: Initialized, Role: FOLLOWER
I20260927 18:40:32.335291 22030 consensus_queue.cc:260] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.335419 22032 consensus_queue.cc:260] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.336050 22013 heartbeater.cc:503] Master 127.6.4.62:44179 was elected leader, sending a full tablet report...
I20260927 18:40:32.336310 22030 ts_tablet_manager.cc:1428] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260927 18:40:32.336321 22032 ts_tablet_manager.cc:1428] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:40:32.336628 21879 heartbeater.cc:503] Master 127.6.4.62:44179 was elected leader, sending a full tablet report...
I20260927 18:40:32.337242 22031 raft_consensus.cc:348] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.337568 22031 raft_consensus.cc:374] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:32.337663 22031 raft_consensus.cc:729] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5df9393f1a4af6b3f4913dcd8080fe, State: Initialized, Role: FOLLOWER
I20260927 18:40:32.338733 22031 consensus_queue.cc:260] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.339303 21745 heartbeater.cc:503] Master 127.6.4.62:44179 was elected leader, sending a full tablet report...
I20260927 18:40:32.339622 22031 ts_tablet_manager.cc:1428] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260927 18:40:32.514735 22014 tablet.cc:2369] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:40:32.550134 21880 tablet.cc:2369] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:40:32.584949 21746 tablet.cc:2369] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:32.638366 22036 raft_consensus.cc:482] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:32.638597 22036 raft_consensus.cc:504] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.639844 22036 leader_election.cc:288] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339), fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859)
I20260927 18:40:32.644690 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.640662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:52579 (local address 127.6.4.1:40859)
0927 18:40:32.640795 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.640800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.640813 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:40:32.640947 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.640949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.640986 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.641077 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:32.641313 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.642004 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.643463 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.644078 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.644207 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.644271 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.644486 (+ 215us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.644559 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:40:32.644690 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.640248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:58483 (local address 127.6.4.3:36339)
0927 18:40:32.640401 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.640407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.640507 (+ 100us) server_negotiation.cc:407] Connection header received
0927 18:40:32.640675 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.640679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.640735 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.640819 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:32.641106 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.641861 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.643463 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.644078 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.644207 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.644271 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.644509 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.644575 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:40:32.644753 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.640145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:36339 (local address 127.6.4.2:58483)
0927 18:40:32.640414 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.640428 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.640556 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.640842 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.640845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.640859 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:32.641000 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.641006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.641976 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.641979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.643336 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.643346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.644206 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.644210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.644356 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.644403 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.644478 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.644577 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":87,"thread_start_us":92,"threads_started":1}
I20260927 18:40:32.644755 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.640505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:40859 (local address 127.6.4.2:52579)
0927 18:40:32.640752 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.640763 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:32.640844 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:32.641105 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:32.641108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:32.641124 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:32.641218 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.641222 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.642094 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.642097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.643336 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:32.643346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.644206 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.644210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:32.644351 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.644394 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:40:32.644472 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:40:32.644577 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":77,"thread_start_us":71,"threads_started":1}
I20260927 18:40:32.645663 21699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4981516f0939451baf16bfb401c1815d" candidate_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" is_pre_election: true
I20260927 18:40:32.645660 21967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4981516f0939451baf16bfb401c1815d" candidate_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3507259df9be4a908c33406f7a8642cc" is_pre_election: true
I20260927 18:40:32.645990 21967 raft_consensus.cc:2454] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd862bbed3a46ad836a7dc8bb0aad6a in term 0.
I20260927 18:40:32.646003 21699 raft_consensus.cc:2454] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd862bbed3a46ad836a7dc8bb0aad6a in term 0.
I20260927 18:40:32.646436 21767 leader_election.cc:302] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc; no voters:
I20260927 18:40:32.646672 22036 raft_consensus.cc:2781] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:32.646751 22036 raft_consensus.cc:482] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:32.646802 22036 raft_consensus.cc:3037] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:32.647816 22036 raft_consensus.cc:504] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.648177 22036 leader_election.cc:288] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [CANDIDATE]: Term 1 election: Requested vote from peers 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339), fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859)
I20260927 18:40:32.648468 21967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4981516f0939451baf16bfb401c1815d" candidate_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3507259df9be4a908c33406f7a8642cc"
I20260927 18:40:32.648527 21699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4981516f0939451baf16bfb401c1815d" candidate_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe"
I20260927 18:40:32.648582 21967 raft_consensus.cc:3037] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:32.648646 21699 raft_consensus.cc:3037] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:32.649601 21699 raft_consensus.cc:2454] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd862bbed3a46ad836a7dc8bb0aad6a in term 1.
I20260927 18:40:32.649601 21967 raft_consensus.cc:2454] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd862bbed3a46ad836a7dc8bb0aad6a in term 1.
I20260927 18:40:32.649854 21767 leader_election.cc:302] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, fb5df9393f1a4af6b3f4913dcd8080fe; no voters:
I20260927 18:40:32.650024 22036 raft_consensus.cc:2781] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:32.650216 22036 raft_consensus.cc:686] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 LEADER]: Becoming Leader. State: Replica: 0bd862bbed3a46ad836a7dc8bb0aad6a, State: Running, Role: LEADER
I20260927 18:40:32.650523 22036 consensus_queue.cc:237] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:32.653600 21552 catalog_manager.cc:5495] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2). New cstate: current_term: 1 leader_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } health_report { overall_health: UNKNOWN } } }
I20260927 18:40:32.686100 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.682909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.6.4.2:41185)
0927 18:40:32.683049 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:32.683053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:32.683066 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:40:32.683141 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:32.683144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:32.683198 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:32.683285 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:32.683452 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.683939 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.684796 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:32.685045 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:32.685119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:32.685363 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:32.685779 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:32.685884 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:32.685914 (+ 30us) server_negotiation.cc:299] Negotiation successful
0927 18:40:32.685971 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:40:32.692075 21967 raft_consensus.cc:1261] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Refusing update from remote peer 0bd862bbed3a46ad836a7dc8bb0aad6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:32.692075 21699 raft_consensus.cc:1261] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Refusing update from remote peer 0bd862bbed3a46ad836a7dc8bb0aad6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:32.692778 22036 consensus_queue.cc:1035] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:32.692987 22041 consensus_queue.cc:1035] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:32.699606 22044 mvcc.cc:205] Tried to move back new op lower bound from 7334029036298825728 to 7334029036138983424. Current Snapshot: MvccSnapshot[applied={T|T < 7334029036298825728}]
I20260927 18:40:32.708366 21552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:32.708568 21552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:32.710961 21552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Sending DeleteTablet for 3 replicas of tablet 4981516f0939451baf16bfb401c1815d
I20260927 18:40:32.711704 21813 tablet_service.cc:1514] Processing DeleteTablet for tablet 4981516f0939451baf16bfb401c1815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:32 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:42732
I20260927 18:40:32.711762 21679 tablet_service.cc:1514] Processing DeleteTablet for tablet 4981516f0939451baf16bfb401c1815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:32 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:45926
I20260927 18:40:32.711817 21947 tablet_service.cc:1514] Processing DeleteTablet for tablet 4981516f0939451baf16bfb401c1815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:32 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:37380
I20260927 18:40:32.712175 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21616
I20260927 18:40:32.713222 22055 tablet_replica.cc:330] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: stopping tablet replica
I20260927 18:40:32.713562 22055 raft_consensus.cc:2229] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:32.713892 22055 raft_consensus.cc:2258] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:32.715322 22055 ts_tablet_manager.cc:1905] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:32.716315 22054 tablet_replica.cc:330] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: stopping tablet replica
I20260927 18:40:32.716662 22054 raft_consensus.cc:2229] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:40:32.717073 22054 raft_consensus.cc:2258] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:32.717981 22055 ts_tablet_manager.cc:1918] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:32.718119 22055 log.cc:1196] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/4981516f0939451baf16bfb401c1815d
I20260927 18:40:32.718483 22055 ts_tablet_manager.cc:1939] T 4981516f0939451baf16bfb401c1815d P 3507259df9be4a908c33406f7a8642cc: Deleting consensus metadata
I20260927 18:40:32.718473 22054 ts_tablet_manager.cc:1905] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:32.719780 21539 catalog_manager.cc:4841] TS 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): tablet 4981516f0939451baf16bfb401c1815d (table test-table [id=914477ea494244b78b71734caf5119c0]) successfully deleted
W20260927 18:40:32.720124 21539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:40859 (error 108)
I20260927 18:40:32.720887 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21750
I20260927 18:40:32.720999 22054 ts_tablet_manager.cc:1918] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:32.721072 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.720761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:40859 (local address 127.0.0.1:45930)
0927 18:40:32.720889 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.720970 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:40859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:40:32.721124 22054 log.cc:1196] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/4981516f0939451baf16bfb401c1815d
W20260927 18:40:32.721741 21539 catalog_manager.cc:4584] TS fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4981516f0939451baf16bfb401c1815d: Network error: Client connection negotiation failed: client connection to 127.6.4.1:40859: connect: Connection refused (error 111)
I20260927 18:40:32.726119 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21884
I20260927 18:40:32.727061 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:32.726252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41185 (local address 127.0.0.1:42758)
0927 18:40:32.726928 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:32.726981 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.2:41185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596}
I20260927 18:40:32.730997 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 21522
I20260927 18:40:52.736743 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44179
--webserver_interface=127.6.4.62
--webserver_port=46287
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:44179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:40:52.848774 22057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:52.849061 22057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:52.849103 22057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:52.853248 22057 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:40:52.853314 22057 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:40:52.853329 22057 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:40:52.853344 22057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:40:52.853363 22057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:52.853379 22057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:52.853398 22057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:40:52.853430 22057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:40:52.858879 22057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:44179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=46287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:52.860045 22057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:52.861094 22057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:52.873919 22064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:52.873950 22063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:52.874061 22057 server_base.cc:1034] running on GCE node
W20260927 18:40:52.873942 22066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:52.874773 22057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:52.875924 22057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:52.877094 22057 hybrid_clock.cc:648] HybridClock initialized: now 1790534452877060 us; error 44 us; skew 500 ppm
Sep 27 18:40:52 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534452, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:52.886667 22057 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:40:52.887702 22057 webserver.cc:460] Webserver started at http://127.6.4.62:46287/ using document root <none> and password file <none>
I20260927 18:40:52.888391 22057 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:52.888479 22057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:52.892575 22057 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:52.894753 22073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:52.896049 22057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 18:40:52.896198 22057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "96bf12e6b1f147b0b44852bad50834fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "77ab79e6172c0f478fb3d979bd058f29"
server_key_iv: "eccda889c4378631204a92e72debd868"
server_key_version: "encryptionkey@0"
I20260927 18:40:52.896792 22057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:52.929224 22057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:52.932477 22057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:52.932643 22057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:52.939622 22057 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:44179
I20260927 18:40:52.939901 22125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:44179 every 8 connection(s)
I20260927 18:40:52.940811 22057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:40:52.943426 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22057
I20260927 18:40:52.943851 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:40859
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:44179
--webserver_port=42227
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:52.945806 22126 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:40:52.948119 22126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap starting.
I20260927 18:40:52.971266 22126 log.cc:824] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:52.982213 22126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:52.982666 22126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap complete.
I20260927 18:40:52.988271 22126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:52.988798 22126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96bf12e6b1f147b0b44852bad50834fe, State: Initialized, Role: FOLLOWER
I20260927 18:40:52.989367 22126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:52.989523 22126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:40:52.989598 22126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:40:52.989691 22126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:40:52.992230 22126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:52.992623 22126 leader_election.cc:302] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96bf12e6b1f147b0b44852bad50834fe; no voters:
I20260927 18:40:52.992985 22126 leader_election.cc:288] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:40:52.993110 22130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:40:52.993355 22130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 LEADER]: Becoming Leader. State: Replica: 96bf12e6b1f147b0b44852bad50834fe, State: Running, Role: LEADER
I20260927 18:40:52.993705 22130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:40:52.994287 22126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:40:52.995116 22132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bf12e6b1f147b0b44852bad50834fe. Latest consensus state: current_term: 2 leader_uuid: "96bf12e6b1f147b0b44852bad50834fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } } }
I20260927 18:40:52.995116 22131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96bf12e6b1f147b0b44852bad50834fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } } }
I20260927 18:40:52.995256 22132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:52.995256 22131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: This master's current role is: LEADER
I20260927 18:40:52.995788 22139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:40:53.000488 22139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:40:53.001281 22139 catalog_manager.cc:1259] Loaded cluster ID: 236d34faf7254f3aa56b28da5d6341c1
I20260927 18:40:53.001380 22139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:40:53.003190 22139 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:40:53.003772 22139 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Loaded TSK: 0
I20260927 18:40:53.021040 22139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Generated new TSK 1
I20260927 18:40:53.021399 22139 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:40:53.087229 22128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:53.087491 22128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:53.087551 22128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:53.091635 22128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:53.091706 22128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:53.091789 22128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:40:53.097349 22128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:40859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=42227
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:53.098393 22128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:53.099413 22128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:53.109067 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:53.109040 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:53.109062 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:53.109210 22128 server_base.cc:1034] running on GCE node
I20260927 18:40:53.109795 22128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:53.110605 22128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:53.111806 22128 hybrid_clock.cc:648] HybridClock initialized: now 1790534453111784 us; error 37 us; skew 500 ppm
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:53.115321 22128 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:40:53.116576 22128 webserver.cc:460] Webserver started at http://127.6.4.1:42227/ using document root <none> and password file <none>
I20260927 18:40:53.117338 22128 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:53.117466 22128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:53.122843 22128 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:40:53.125679 22163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:53.127223 22128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:40:53.127364 22128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "fb5df9393f1a4af6b3f4913dcd8080fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "bd7a59c830bfcad54eecb62268040d9c"
server_key_iv: "d50d31202211c01cd0706bebf1122b75"
server_key_version: "encryptionkey@0"
I20260927 18:40:53.128137 22128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:53.147547 22128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:53.151535 22128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:53.151748 22128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:53.152472 22128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:53.153752 22170 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:40:53.158064 22128 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:40:53.158162 22128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260927 18:40:53.158227 22128 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:40:53.160501 22128 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:40:53.160573 22128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:53.160833 22170 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap starting.
I20260927 18:40:53.170001 22128 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:40859
I20260927 18:40:53.170079 22277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:40859 every 8 connection(s)
I20260927 18:40:53.170991 22128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:40:53.171376 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22128
I20260927 18:40:53.171875 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:41185
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:44179
--webserver_port=35675
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:40:53.181285 22170 log.cc:824] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:53.188501 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.173085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:44907)
0927 18:40:53.174037 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.174107 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.175195 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.176447 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.176455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.176872 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:53.177357 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.177376 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.178675 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.178678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.179566 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.179576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.180316 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.180352 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.180548 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.181104 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:53.181120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:53.182704 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:53.185211 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.185218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.185231 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.185549 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.185905 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.185908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.185919 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.186051 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.186703 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:53.186710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:53.186948 (+ 238us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.187106 (+ 158us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.187487 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":122,"threads_started":1}
I20260927 18:40:53.188488 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.173062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44907 (local address 127.6.4.62:44179)
0927 18:40:53.173667 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.173689 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.174250 (+ 561us) server_negotiation.cc:407] Connection header received
0927 18:40:53.175640 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.175652 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.176016 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.176365 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:53.177538 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.178522 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.179687 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.180218 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.180373 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.182896 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:53.182901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:53.182917 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:53.182945 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:53.185090 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.185664 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.185670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.185676 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.185806 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.186138 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.186141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.186143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.186585 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:53.186698 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.187264 (+ 566us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.187572 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":112,"threads_started":1}
I20260927 18:40:53.190231 22170 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:40:53.190686 22170 tablet_bootstrap.cc:492] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap complete.
I20260927 18:40:53.191030 22278 heartbeater.cc:347] Connected to a master server at 127.6.4.62:44179
I20260927 18:40:53.191256 22278 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:53.191752 22278 heartbeater.cc:511] Master 127.6.4.62:44179 requested a full tablet report, sending...
I20260927 18:40:53.192082 22170 ts_tablet_manager.cc:1397] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.009s
I20260927 18:40:53.194202 22089 ts_manager.cc:194] Registered new tserver with Master: fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859)
I20260927 18:40:53.195087 22170 raft_consensus.cc:348] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.195596 22170 raft_consensus.cc:729] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5df9393f1a4af6b3f4913dcd8080fe, State: Initialized, Role: FOLLOWER
I20260927 18:40:53.196139 22170 consensus_queue.cc:260] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.197160 22170 ts_tablet_manager.cc:1428] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260927 18:40:53.197852 22089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:44907
I20260927 18:40:53.200474 22278 heartbeater.cc:503] Master 127.6.4.62:44179 was elected leader, sending a full tablet report...
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534452, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.1@KRBTEST.COM
I20260927 18:40:53.207918 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.197965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60906 (local address 127.6.4.1:40859)
0927 18:40:53.198260 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.198272 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.198298 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:40:53.198389 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.198394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.198604 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.198731 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:53.199514 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.200581 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.201813 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.202226 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.202307 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.204663 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:53.204671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:53.204681 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:53.204703 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:53.206089 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.206578 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.206582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.206583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.206660 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.206904 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.206906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.206907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.207135 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:53.207231 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.207645 (+ 414us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.207791 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260927 18:40:53.207924 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.197211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:40859 (local address 127.0.0.1:60906)
0927 18:40:53.197688 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.197745 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.198240 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.198796 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.198802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.199232 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:53.199384 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.199392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.200732 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.200736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.201686 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.201698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.202319 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.202336 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.202447 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.202970 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:53.202984 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:53.204532 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:53.206192 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.206201 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.206207 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.206479 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.206737 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.206740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.206741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.206831 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.207231 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:53.207238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:53.207372 (+ 134us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.207537 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.207747 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":101,"threads_started":1}
I20260927 18:40:53.209088 22212 tablet_service.cc:1514] Processing DeleteTablet for tablet 4981516f0939451baf16bfb401c1815d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:32 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:60906
I20260927 18:40:53.209923 22287 tablet_replica.cc:330] stopping tablet replica
I20260927 18:40:53.210210 22287 raft_consensus.cc:2229] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:53.210443 22287 raft_consensus.cc:2258] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:53.211392 22287 ts_tablet_manager.cc:1905] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:53.214888 22287 ts_tablet_manager.cc:1918] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:53.214987 22287 log.cc:1196] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal/wals/4981516f0939451baf16bfb401c1815d
I20260927 18:40:53.215201 22287 ts_tablet_manager.cc:1939] T 4981516f0939451baf16bfb401c1815d P fb5df9393f1a4af6b3f4913dcd8080fe: Deleting consensus metadata
I20260927 18:40:53.217061 22075 catalog_manager.cc:4841] TS fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859): tablet 4981516f0939451baf16bfb401c1815d (table test-table [id=914477ea494244b78b71734caf5119c0]) successfully deleted
W20260927 18:40:53.299844 22281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:53.300254 22281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:53.300318 22281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:53.304786 22281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:53.304862 22281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:53.304924 22281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:40:53.310833 22281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:41185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=35675
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:53.312146 22281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:53.313397 22281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:53.324715 22292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:53.324805 22281 server_base.cc:1034] running on GCE node
W20260927 18:40:53.324718 22293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:53.324715 22295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:53.325562 22281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:53.326380 22281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:53.327601 22281 hybrid_clock.cc:648] HybridClock initialized: now 1790534453327576 us; error 48 us; skew 500 ppm
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:53.331243 22281 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:40:53.332506 22281 webserver.cc:460] Webserver started at http://127.6.4.2:35675/ using document root <none> and password file <none>
I20260927 18:40:53.333268 22281 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:53.333391 22281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:53.338317 22281 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:40:53.341269 22302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:53.342931 22281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:40:53.343117 22281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "aba9ede6f4cfb54913a4ee0d79daa226"
server_key_iv: "f1db9507cb733f676432544839ece482"
server_key_version: "encryptionkey@0"
I20260927 18:40:53.343807 22281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:53.370831 22281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:53.374285 22281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:53.374507 22281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:53.375166 22281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:53.376371 22309 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260927 18:40:53.380421 22309 ts_tablet_manager.cc:1852] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260927 18:40:53.380810 22309 ts_tablet_manager.cc:1905] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:53.384724 22309 ts_tablet_manager.cc:1918] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:53.384840 22309 log.cc:1196] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/4981516f0939451baf16bfb401c1815d
I20260927 18:40:53.385083 22309 ts_tablet_manager.cc:1939] T 4981516f0939451baf16bfb401c1815d P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting consensus metadata
I20260927 18:40:53.385392 22281 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260927 18:40:53.385454 22281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260927 18:40:53.385525 22281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:53.385555 22281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:53.394086 22281 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:41185
I20260927 18:40:53.394157 22416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:41185 every 8 connection(s)
I20260927 18:40:53.395021 22281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:40:53.402737 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22281
I20260927 18:40:53.403168 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:36339
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:44179
--webserver_port=39723
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:53.407683 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.396823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:36963 (local address 127.6.4.62:44179)
0927 18:40:53.396961 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.396973 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.397827 (+ 854us) server_negotiation.cc:407] Connection header received
0927 18:40:53.398698 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.398701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.398747 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.398818 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:53.399919 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.400518 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.401684 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.401912 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.402019 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.404385 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:53.404389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:53.404391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:53.404421 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:53.405777 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.406254 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.406257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.406259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.406333 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.406639 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.406642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.406644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.406826 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:53.406910 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.407390 (+ 480us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.407520 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:40:53.408542 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.397256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:36963)
0927 18:40:53.397685 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.397728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.398522 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.399000 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.399006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.399340 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:53.399802 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.399815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.400630 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.400633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.401585 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.401594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.401978 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.402023 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.402159 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.402718 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:53.402734 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:53.404182 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:53.405886 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.405892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.405908 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.406161 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.406412 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.406416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.406418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.406555 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.406922 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:53.406929 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:53.407171 (+ 242us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.407333 (+ 162us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.407663 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":100,"threads_started":1}
I20260927 18:40:53.409802 22417 heartbeater.cc:347] Connected to a master server at 127.6.4.62:44179
I20260927 18:40:53.410053 22417 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:53.410521 22417 heartbeater.cc:511] Master 127.6.4.62:44179 requested a full tablet report, sending...
I20260927 18:40:53.411599 22089 ts_manager.cc:194] Registered new tserver with Master: 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185)
I20260927 18:40:53.412395 22089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:36963
W20260927 18:40:53.513236 22420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:40:53.513517 22420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:40:53.513561 22420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:40:53.517699 22420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:40:53.517771 22420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:40:53.517832 22420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:40:53.523233 22420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:36339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=39723
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:40:53.524353 22420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:40:53.525328 22420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:40:53.534850 22425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:53.534960 22420 server_base.cc:1034] running on GCE node
W20260927 18:40:53.534878 22428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:40:53.534850 22426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:40:53.535565 22420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:40:53.536221 22420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:40:53.537372 22420 hybrid_clock.cc:648] HybridClock initialized: now 1790534453537351 us; error 41 us; skew 500 ppm
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:40:53.540017 22420 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:40:53.541036 22420 webserver.cc:460] Webserver started at http://127.6.4.3:39723/ using document root <none> and password file <none>
I20260927 18:40:53.541597 22420 fs_manager.cc:359] Metadata directory not provided
I20260927 18:40:53.541675 22420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:40:53.545526 22420 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:40:53.547626 22435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:53.548979 22420 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:40:53.549086 22420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "3507259df9be4a908c33406f7a8642cc"
format_stamp: "Formatted at 2026-09-27 18:40:32 on dist-test-slave-bzzj"
server_key: "68830f3fa28a00eacb89a67227e52a31"
server_key_iv: "c9076fb3d4d1cc5ef93011b02eeb0572"
server_key_version: "encryptionkey@0"
I20260927 18:40:53.549646 22420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:40:53.561563 22420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:40:53.564736 22420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:40:53.564917 22420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:40:53.565502 22420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:40:53.566309 22420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:40:53.566354 22420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:53.566406 22420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:40:53.566432 22420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:40:53.574779 22420 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:36339
I20260927 18:40:53.574801 22548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:36339 every 8 connection(s)
I20260927 18:40:53.575798 22420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:40:53.579540 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22420
Sep 27 18:40:53 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:40:53.588267 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.580304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.6.4.62:44179)
0927 18:40:53.580685 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.580689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.580702 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:40:53.580844 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.580848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.580886 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.580960 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:53.581692 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.582225 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.583091 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.583438 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.583512 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.585020 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:53.585024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:53.585026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:53.585053 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:53.586670 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.587045 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.587049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.587051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.587130 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.587374 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.587378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.587380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.587561 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:53.587749 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.587918 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.588044 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":314,"thread_start_us":79,"threads_started":1}
I20260927 18:40:53.589700 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.577668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:60261 (local address 127.6.4.62:44179)
0927 18:40:53.577835 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.577839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.578674 (+ 835us) server_negotiation.cc:407] Connection header received
0927 18:40:53.580174 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.580177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.580238 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.580320 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:53.581546 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.582108 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.583353 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.583706 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.583803 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.586351 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:53.586354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:53.586356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:53.586389 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:53.587616 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.588123 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.588134 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.588135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.588233 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.588607 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.588610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.588612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.588813 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:53.588895 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.589449 (+ 554us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.589555 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":65}
I20260927 18:40:53.590654 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.578069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:60261)
0927 18:40:53.578525 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.578568 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.579905 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.580548 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.580555 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.580914 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:40:53.581396 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.581408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.582225 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.582228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.583227 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.583237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.583811 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.583858 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.584054 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.584572 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:40:53.584588 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:40:53.586184 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:40:53.587722 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.587728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.587746 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.588000 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.588307 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:40:53.588310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:40:53.588312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:40:53.588500 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:40:53.588907 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:40:53.588914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:40:53.589200 (+ 286us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.589400 (+ 200us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.589756 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260927 18:40:53.591706 22549 heartbeater.cc:347] Connected to a master server at 127.6.4.62:44179
I20260927 18:40:53.591955 22549 heartbeater.cc:464] Registering TS with master...
I20260927 18:40:53.592453 22549 heartbeater.cc:511] Master 127.6.4.62:44179 requested a full tablet report, sending...
I20260927 18:40:53.593395 22089 ts_manager.cc:194] Registered new tserver with Master: 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
I20260927 18:40:53.593983 22089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:60261
I20260927 18:40:53.601869 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:40:53.606186 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.602478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.6.4.62:44179)
0927 18:40:53.602588 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.602592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.602806 (+ 214us) server_negotiation.cc:407] Connection header received
0927 18:40:53.603002 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.603004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.603110 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.603200 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:53.603443 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.603948 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.605013 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.605236 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.605330 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.605637 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:53.605783 (+ 146us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 18:40:53.606047 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59076: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260927 18:40:53.606261 22280 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59076: token expired
W20260927 18:40:53.606587 22018 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.4.62:44179)
I20260927 18:40:53.613127 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.606926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59088 (local address 127.6.4.62:44179)
0927 18:40:53.607029 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.607032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.607044 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:40:53.607156 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.607158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.607194 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.607254 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:40:53.607789 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.608294 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.609022 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.609224 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.609296 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.610490 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:40:53.610493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:40:53.610495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:40:53.610516 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:40:53.611709 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.612064 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.612067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.612068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.612180 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:40:53.612433 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:40:53.612436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:40:53.612437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:40:53.612613 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:40:53.612717 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.612883 (+ 166us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.612971 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:40:53.615638 22089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:40:53.619079 22089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:40:53.627125 22212 tablet_service.cc:1467] Processing CreateTablet for tablet fa8b29d6c5b8495c9b89b04eeaebdf79 (DEFAULT_TABLE table=test-table [id=3c36923da2444a7bb27eb1564124c048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:53.627787 22212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8b29d6c5b8495c9b89b04eeaebdf79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:53.630568 22170 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap starting.
I20260927 18:40:53.630895 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.626154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:36339 (local address 127.0.0.1:40072)
0927 18:40:53.626373 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.626386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.626470 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.627244 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.627247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.627266 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:53.627367 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.627374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.628680 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.628683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.629460 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.629469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.630458 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.630461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.630551 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.630604 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.630673 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.630753 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260927 18:40:53.631410 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.626327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40072 (local address 127.6.4.3:36339)
0927 18:40:53.626766 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.626779 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.626801 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:40:53.626874 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.626881 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.627101 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.627260 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:53.627475 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.628584 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.629576 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.630328 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.630436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.630600 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.631147 (+ 547us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.631262 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":150,"threads_started":1}
I20260927 18:40:53.631436 22170 tablet_bootstrap.cc:654] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:53.632906 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.625690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41185 (local address 127.0.0.1:47004)
0927 18:40:53.625849 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.625866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.625950 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.626787 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.626790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.626810 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:53.626926 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.626933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.628538 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.628541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.629648 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.629659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.632455 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.632459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.632559 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.632588 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.632670 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.632773 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:40:53.632972 22170 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: No bootstrap required, opened a new log
I20260927 18:40:53.633054 22170 ts_tablet_manager.cc:1397] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:40:53.633527 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.625908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.6.4.2:41185)
0927 18:40:53.626214 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.626227 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.626252 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:40:53.626333 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.626339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.626586 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.626805 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:53.627031 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.628426 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.629793 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.632313 (+ 2520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.633088 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.633243 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.633300 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.633397 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
I20260927 18:40:53.633518 22170 raft_consensus.cc:348] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.633627 22170 raft_consensus.cc:374] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:53.633661 22170 raft_consensus.cc:729] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5df9393f1a4af6b3f4913dcd8080fe, State: Initialized, Role: FOLLOWER
I20260927 18:40:53.633780 22170 consensus_queue.cc:260] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.634039 22170 ts_tablet_manager.cc:1428] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:40:53.634770 22483 tablet_service.cc:1467] Processing CreateTablet for tablet fa8b29d6c5b8495c9b89b04eeaebdf79 (DEFAULT_TABLE table=test-table [id=3c36923da2444a7bb27eb1564124c048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:53.635926 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8b29d6c5b8495c9b89b04eeaebdf79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:53.635762 22351 tablet_service.cc:1467] Processing CreateTablet for tablet fa8b29d6c5b8495c9b89b04eeaebdf79 (DEFAULT_TABLE table=test-table [id=3c36923da2444a7bb27eb1564124c048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:40:53.636770 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa8b29d6c5b8495c9b89b04eeaebdf79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:40:53.641671 22309 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Bootstrap starting.
I20260927 18:40:53.643579 22309 tablet_bootstrap.cc:654] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:53.643692 22562 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Bootstrap starting.
I20260927 18:40:53.644657 22309 log.cc:824] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:53.645717 22562 tablet_bootstrap.cc:654] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Neither blocks nor log segments found. Creating new log.
I20260927 18:40:53.646486 22309 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: No bootstrap required, opened a new log
I20260927 18:40:53.646646 22309 ts_tablet_manager.cc:1397] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:40:53.646710 22562 log.cc:824] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:40:53.648418 22562 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: No bootstrap required, opened a new log
I20260927 18:40:53.648603 22562 ts_tablet_manager.cc:1397] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:40:53.649822 22309 raft_consensus.cc:348] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.650048 22309 raft_consensus.cc:374] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:53.650094 22309 raft_consensus.cc:729] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd862bbed3a46ad836a7dc8bb0aad6a, State: Initialized, Role: FOLLOWER
I20260927 18:40:53.650507 22309 consensus_queue.cc:260] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.651094 22417 heartbeater.cc:503] Master 127.6.4.62:44179 was elected leader, sending a full tablet report...
I20260927 18:40:53.651470 22309 ts_tablet_manager.cc:1428] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:40:53.651556 22562 raft_consensus.cc:348] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.651773 22562 raft_consensus.cc:374] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:40:53.651834 22562 raft_consensus.cc:729] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3507259df9be4a908c33406f7a8642cc, State: Initialized, Role: FOLLOWER
I20260927 18:40:53.652240 22562 consensus_queue.cc:260] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.652809 22549 heartbeater.cc:503] Master 127.6.4.62:44179 was elected leader, sending a full tablet report...
I20260927 18:40:53.653142 22562 ts_tablet_manager.cc:1428] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260927 18:40:53.657229 22284 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:40:53.657402 22284 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.658383 22284 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
I20260927 18:40:53.662017 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.658829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:53813 (local address 127.6.4.3:36339)
0927 18:40:53.658931 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.658935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.658949 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:40:53.659075 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.659079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.659121 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.659178 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:53.659426 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.659916 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.661062 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.661469 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.661568 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.661628 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.661828 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.661902 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:40:53.662036 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.658602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:36339 (local address 127.6.4.1:53813)
0927 18:40:53.658857 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.658870 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.658960 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.659200 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.659203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.659217 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:53.659337 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.659342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.660013 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.660017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.660948 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.660957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.661580 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.661583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.661681 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.661727 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.661809 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.661885 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":17,"thread_start_us":71,"threads_started":1}
I20260927 18:40:53.662111 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.658690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44283 (local address 127.6.4.2:41185)
0927 18:40:53.658834 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.658839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.658941 (+ 102us) server_negotiation.cc:407] Connection header received
0927 18:40:53.659087 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.659090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.659131 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.659193 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:40:53.659434 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.659954 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.661072 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.661483 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.661574 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.661632 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.661943 (+ 311us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.662012 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:40:53.662441 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.658545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41185 (local address 127.6.4.1:44283)
0927 18:40:53.658858 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.658871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:40:53.658967 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:40:53.659210 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:40:53.659213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:40:53.659227 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:40:53.659341 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.659346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.660070 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.660073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.660948 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:40:53.660957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.661706 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.661709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:40:53.661812 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.661835 (+ 23us) client_negotiation.cc:769] Sending connection context
0927 18:40:53.662229 (+ 394us) client_negotiation.cc:240] Negotiation successful
0927 18:40:53.662304 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":25}
I20260927 18:40:53.662667 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8b29d6c5b8495c9b89b04eeaebdf79" candidate_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3507259df9be4a908c33406f7a8642cc" is_pre_election: true
I20260927 18:40:53.662983 22503 raft_consensus.cc:2454] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb5df9393f1a4af6b3f4913dcd8080fe in term 0.
I20260927 18:40:53.663096 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8b29d6c5b8495c9b89b04eeaebdf79" candidate_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" is_pre_election: true
I20260927 18:40:53.663374 22371 raft_consensus.cc:2454] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb5df9393f1a4af6b3f4913dcd8080fe in term 0.
I20260927 18:40:53.663455 22165 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3507259df9be4a908c33406f7a8642cc, fb5df9393f1a4af6b3f4913dcd8080fe; no voters:
I20260927 18:40:53.663726 22284 raft_consensus.cc:2781] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:40:53.663793 22284 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:40:53.663822 22284 raft_consensus.cc:3037] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:53.664896 22284 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.665221 22284 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 1 election: Requested vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
I20260927 18:40:53.665576 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8b29d6c5b8495c9b89b04eeaebdf79" candidate_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3507259df9be4a908c33406f7a8642cc"
I20260927 18:40:53.665576 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa8b29d6c5b8495c9b89b04eeaebdf79" candidate_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a"
I20260927 18:40:53.665705 22503 raft_consensus.cc:3037] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:53.665705 22371 raft_consensus.cc:3037] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:40:53.666952 22371 raft_consensus.cc:2454] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb5df9393f1a4af6b3f4913dcd8080fe in term 1.
I20260927 18:40:53.666952 22503 raft_consensus.cc:2454] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb5df9393f1a4af6b3f4913dcd8080fe in term 1.
I20260927 18:40:53.667263 22165 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc, fb5df9393f1a4af6b3f4913dcd8080fe; no voters:
I20260927 18:40:53.667428 22284 raft_consensus.cc:2781] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:40:53.667665 22284 raft_consensus.cc:686] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 LEADER]: Becoming Leader. State: Replica: fb5df9393f1a4af6b3f4913dcd8080fe, State: Running, Role: LEADER
I20260927 18:40:53.667953 22284 consensus_queue.cc:237] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:40:53.670886 22089 catalog_manager.cc:5495] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe reported cstate change: term changed from 0 to 1, leader changed from <none> to fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1). New cstate: current_term: 1 leader_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } health_report { overall_health: HEALTHY } } }
W20260927 18:40:53.672328 22279 tablet.cc:2369] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:40:53.690119 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.686942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60914 (local address 127.6.4.1:40859)
0927 18:40:53.687070 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:40:53.687073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:40:53.687085 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:40:53.687161 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:40:53.687164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:40:53.687215 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:40:53.687308 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:40:53.687490 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.688002 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.688889 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:40:53.689134 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:40:53.689219 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:40:53.689397 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:40:53.689834 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:40:53.689912 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:40:53.689953 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 18:40:53.690005 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:40:53.693917 22503 raft_consensus.cc:1261] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Refusing update from remote peer fb5df9393f1a4af6b3f4913dcd8080fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:53.693931 22371 raft_consensus.cc:1261] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Refusing update from remote peer fb5df9393f1a4af6b3f4913dcd8080fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:40:53.694626 22568 consensus_queue.cc:1035] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:53.694821 22284 consensus_queue.cc:1035] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:40:53.704454 22573 mvcc.cc:205] Tried to move back new op lower bound from 7334029122324283392 to 7334029122226798592. Current Snapshot: MvccSnapshot[applied={T|T < 7334029122324283392}]
I20260927 18:40:53.704649 22572 mvcc.cc:205] Tried to move back new op lower bound from 7334029122324283392 to 7334029122226798592. Current Snapshot: MvccSnapshot[applied={T|T < 7334029122324283392}]
I20260927 18:40:53.709860 22090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:53.710021 22090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:40:53.712282 22090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Sending DeleteTablet for 3 replicas of tablet fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:40:53.712936 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet fa8b29d6c5b8495c9b89b04eeaebdf79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:53 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:47004
I20260927 18:40:53.713084 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22128
I20260927 18:40:53.713109 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet fa8b29d6c5b8495c9b89b04eeaebdf79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:40:53 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:40072
I20260927 18:40:53.713837 22581 tablet_replica.cc:330] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: stopping tablet replica
I20260927 18:40:53.714123 22581 raft_consensus.cc:2229] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:53.714406 22581 raft_consensus.cc:2258] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:53.715499 22580 tablet_replica.cc:330] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: stopping tablet replica
I20260927 18:40:53.715835 22580 raft_consensus.cc:2229] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:40:53.716130 22580 raft_consensus.cc:2258] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:40:53.716523 22581 ts_tablet_manager.cc:1905] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:53.717367 22580 ts_tablet_manager.cc:1905] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:40:53.719182 22581 ts_tablet_manager.cc:1918] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:53.719280 22581 log.cc:1196] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:40:53.719357 22580 ts_tablet_manager.cc:1918] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:40:53.719429 22580 log.cc:1196] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:40:53.719620 22581 ts_tablet_manager.cc:1939] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 3507259df9be4a908c33406f7a8642cc: Deleting consensus metadata
I20260927 18:40:53.719678 22580 ts_tablet_manager.cc:1939] T fa8b29d6c5b8495c9b89b04eeaebdf79 P 0bd862bbed3a46ad836a7dc8bb0aad6a: Deleting consensus metadata
W20260927 18:40:53.720504 22075 connection.cc:535] client connection to 127.6.4.1:40859 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 18:40:53.720660 22075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:40:53.720709 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22281
I20260927 18:40:53.721083 22075 catalog_manager.cc:4841] TS 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): tablet fa8b29d6c5b8495c9b89b04eeaebdf79 (table test-table [id=3c36923da2444a7bb27eb1564124c048]) successfully deleted
I20260927 18:40:53.722409 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.722102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:40859 (local address 127.0.0.1:60916)
0927 18:40:53.722250 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.722324 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:40859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260927 18:40:53.722575 22075 catalog_manager.cc:4584] TS fb5df9393f1a4af6b3f4913dcd8080fe (127.6.4.1:40859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa8b29d6c5b8495c9b89b04eeaebdf79: Network error: Client connection negotiation failed: client connection to 127.6.4.1:40859: connect: Connection refused (error 111)
I20260927 18:40:53.726419 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22420
I20260927 18:40:53.728261 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:40:53.726523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41185 (local address 127.0.0.1:47014)
0927 18:40:53.728121 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0927 18:40:53.728179 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.2:41185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1494}
I20260927 18:40:53.731256 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22057
I20260927 18:41:13.737021 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44179
--webserver_interface=127.6.4.62
--webserver_port=46287
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:44179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:13.846868 22583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:13.847177 22583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:13.847216 22583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:13.851274 22583 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:41:13.851344 22583 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:41:13.851359 22583 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:41:13.851372 22583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:13.851389 22583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:13.851408 22583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:13.851437 22583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:13.851464 22583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:13.857393 22583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:44179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=46287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:13.858839 22583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:13.860312 22583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:13.869959 22589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:13.870110 22583 server_base.cc:1034] running on GCE node
W20260927 18:41:13.870095 22588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:13.870621 22591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:13.871129 22583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:13.872311 22583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:13.873479 22583 hybrid_clock.cc:648] HybridClock initialized: now 1790534473873451 us; error 43 us; skew 500 ppm
Sep 27 18:41:13 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:13.882556 22583 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:13.883744 22583 webserver.cc:460] Webserver started at http://127.6.4.62:46287/ using document root <none> and password file <none>
I20260927 18:41:13.884516 22583 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:13.884599 22583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:13.890136 22583 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260927 18:41:13.893590 22598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:13.895350 22583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:13.895546 22583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "96bf12e6b1f147b0b44852bad50834fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "77ab79e6172c0f478fb3d979bd058f29"
server_key_iv: "eccda889c4378631204a92e72debd868"
server_key_version: "encryptionkey@0"
I20260927 18:41:13.896412 22583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:13.916075 22583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:13.920120 22583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:13.920322 22583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:13.927572 22583 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:44179
I20260927 18:41:13.927824 22650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:44179 every 8 connection(s)
I20260927 18:41:13.928712 22583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:13.932914 22651 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:41:13.933501 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22583
I20260927 18:41:13.933915 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:40859
--local_ip_for_outbound_sockets=127.6.4.1
--tserver_master_addrs=127.6.4.62:44179
--webserver_port=42227
--webserver_interface=127.6.4.1
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:13.934620 22651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap starting.
I20260927 18:41:13.951354 22651 log.cc:824] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:13.962417 22651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:13.962770 22651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: Bootstrap complete.
I20260927 18:41:13.966543 22651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:41:13.966921 22651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96bf12e6b1f147b0b44852bad50834fe, State: Initialized, Role: FOLLOWER
I20260927 18:41:13.967371 22651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:41:13.967489 22651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:13.967554 22651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:13.967643 22651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:41:13.969540 22651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:41:13.969848 22651 leader_election.cc:302] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96bf12e6b1f147b0b44852bad50834fe; no voters:
I20260927 18:41:13.970145 22651 leader_election.cc:288] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 18:41:13.970243 22655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:41:13.970429 22655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [term 3 LEADER]: Becoming Leader. State: Replica: 96bf12e6b1f147b0b44852bad50834fe, State: Running, Role: LEADER
I20260927 18:41:13.970770 22655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } }
I20260927 18:41:13.971387 22651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:13.971735 22655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bf12e6b1f147b0b44852bad50834fe. Latest consensus state: current_term: 3 leader_uuid: "96bf12e6b1f147b0b44852bad50834fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } } }
I20260927 18:41:13.971920 22655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:13.972205 22656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "96bf12e6b1f147b0b44852bad50834fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96bf12e6b1f147b0b44852bad50834fe" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44179 } } }
I20260927 18:41:13.972375 22656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:13.976572 22671 catalog_manager.cc:1259] Loaded cluster ID: 236d34faf7254f3aa56b28da5d6341c1
I20260927 18:41:13.976642 22671 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: loading cluster ID for follower catalog manager: success
I20260927 18:41:13.979229 22671 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 96bf12e6b1f147b0b44852bad50834fe: acquiring CA information for follower catalog manager: success
F20260927 18:41:13.980492 22671 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdf579519ad google::LogMessage::Fail() at ??:0
@ 0x7fdf5795488a google::LogMessage::SendToLog() at ??:0
@ 0x7fdf579514de google::LogMessage::Flush() at ??:0
@ 0x7fdf57952a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdf5a61518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fdf622b6989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fdf62293023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fdf6229354b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fdf6228c652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fdf6228b9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fdf622d2541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fdf59b94512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdf5b1d86db start_thread at ??:0
@ 0x7fdf5644f61f clone at ??:0
W20260927 18:41:14.074303 22653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:14.074661 22653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:14.074712 22653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:14.079002 22653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:14.079070 22653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:14.079133 22653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:14.084559 22653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:40859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=42227
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:14.085695 22653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:14.086735 22653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:14.096256 22691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:14.096339 22692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:14.096272 22694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:14.096545 22653 server_base.cc:1034] running on GCE node
I20260927 18:41:14.097043 22653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:14.097864 22653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:14.099062 22653 hybrid_clock.cc:648] HybridClock initialized: now 1790534474099045 us; error 30 us; skew 500 ppm
Sep 27 18:41:14 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:14.102430 22653 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:41:14.103545 22653 webserver.cc:460] Webserver started at http://127.6.4.1:42227/ using document root <none> and password file <none>
I20260927 18:41:14.104327 22653 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:14.104425 22653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:14.109594 22653 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:14.112314 22701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.113867 22653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:14.113996 22653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "fb5df9393f1a4af6b3f4913dcd8080fe"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "bd7a59c830bfcad54eecb62268040d9c"
server_key_iv: "d50d31202211c01cd0706bebf1122b75"
server_key_version: "encryptionkey@0"
I20260927 18:41:14.114729 22653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:14.141791 22653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:14.145887 22653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:14.146095 22653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:14.146811 22653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:14.148206 22708 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:41:14.152467 22653 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:41:14.152552 22653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260927 18:41:14.152623 22653 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:41:14.155006 22653 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:41:14.155072 22653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260927 18:41:14.155364 22708 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap starting.
I20260927 18:41:14.165850 22653 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:40859
I20260927 18:41:14.166203 22815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:40859 every 8 connection(s)
I20260927 18:41:14.167114 22653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:14.171150 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22653
I20260927 18:41:14.171659 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:41185
--local_ip_for_outbound_sockets=127.6.4.2
--tserver_master_addrs=127.6.4.62:44179
--webserver_port=35675
--webserver_interface=127.6.4.2
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:14.175127 22708 log.cc:824] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:14.184707 22708 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:41:14.185153 22708 tablet_bootstrap.cc:492] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Bootstrap complete.
I20260927 18:41:14.186447 22708 ts_tablet_manager.cc:1397] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent bootstrapping tablet: real 0.031s user 0.028s sys 0.000s
I20260927 18:41:14.189706 22708 raft_consensus.cc:348] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:14.190274 22708 raft_consensus.cc:729] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5df9393f1a4af6b3f4913dcd8080fe, State: Initialized, Role: FOLLOWER
I20260927 18:41:14.190798 22708 consensus_queue.cc:260] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:14.191905 22708 ts_tablet_manager.cc:1428] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260927 18:41:14.198212 22818 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:41:14.169530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:33581)
0927 18:41:14.170160 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.170227 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:14.171461 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:14.197446 (+ 25985us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":140,"threads_started":1}
W20260927 18:41:14.198691 22816 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:44179 (0 consecutive failures): Network error: Failed to ping master at 127.6.4.62:44179: Client connection negotiation failed: client connection to 127.6.4.62:44179: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:41:14.199406 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.199162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:45519)
0927 18:41:14.199289 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.199352 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:41:14.199983 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.199794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:53167)
0927 18:41:14.199895 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.199940 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260927 18:41:14.200158 22816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:41:14.288926 22819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:14.289207 22819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:14.289268 22819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:14.293331 22819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:14.293399 22819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:14.293479 22819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:14.298884 22819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:41185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=35675
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:14.299957 22819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:14.301026 22819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:14.310354 22827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:14.310472 22819 server_base.cc:1034] running on GCE node
W20260927 18:41:14.310369 22826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:14.310350 22829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:14.311144 22819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:14.311921 22819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:14.313086 22819 hybrid_clock.cc:648] HybridClock initialized: now 1790534474313075 us; error 49 us; skew 500 ppm
Sep 27 18:41:14 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:14.315776 22819 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:41:14.316670 22819 webserver.cc:460] Webserver started at http://127.6.4.2:35675/ using document root <none> and password file <none>
I20260927 18:41:14.317205 22819 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:14.317279 22819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:14.321199 22819 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:14.323189 22836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.324420 22819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:14.324529 22819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a"
format_stamp: "Formatted at 2026-09-27 18:40:31 on dist-test-slave-bzzj"
server_key: "aba9ede6f4cfb54913a4ee0d79daa226"
server_key_iv: "f1db9507cb733f676432544839ece482"
server_key_version: "encryptionkey@0"
I20260927 18:41:14.325073 22819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:14.352880 22819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:14.355764 22819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:14.355952 22819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:14.356575 22819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:14.357395 22819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:14.357451 22819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.357506 22819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:14.357548 22819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.365984 22819 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:41185
I20260927 18:41:14.366034 22949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:41185 every 8 connection(s)
I20260927 18:41:14.366969 22819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:14.370100 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22819
I20260927 18:41:14.370666 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:36339
--local_ip_for_outbound_sockets=127.6.4.3
--tserver_master_addrs=127.6.4.62:44179
--webserver_port=39723
--webserver_interface=127.6.4.3
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:41:14.371464 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.369306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:33823)
0927 18:41:14.369889 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.370534 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":151,"threads_started":1}
W20260927 18:41:14.372010 22950 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:44179 (0 consecutive failures): Network error: Failed to ping master at 127.6.4.62:44179: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
I20260927 18:41:14.372757 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.372506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:42137)
0927 18:41:14.372636 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.372693 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 18:41:14.373842 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.373284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:41721)
0927 18:41:14.373714 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.373781 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340}
W20260927 18:41:14.373970 22950 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:41:14.481282 22953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:14.481551 22953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:14.481611 22953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:14.485764 22953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:14.485953 22953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:14.486094 22953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:14.491590 22953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:36339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=39723
--tserver_master_addrs=127.6.4.62:44179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:14.492718 22953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:14.493793 22953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:14.503178 22961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:14.503232 22959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:14.503284 22958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:14.503369 22953 server_base.cc:1034] running on GCE node
I20260927 18:41:14.503769 22953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:14.504503 22953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:14.505734 22953 hybrid_clock.cc:648] HybridClock initialized: now 1790534474505712 us; error 35 us; skew 500 ppm
Sep 27 18:41:14 dist-test-slave-bzzj krb5kdc[21509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:14.508507 22953 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:41:14.509338 22953 webserver.cc:460] Webserver started at http://127.6.4.3:39723/ using document root <none> and password file <none>
I20260927 18:41:14.509855 22953 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:14.509910 22953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:14.513713 22953 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:14.515769 22968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.517064 22953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:14.517161 22953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "3507259df9be4a908c33406f7a8642cc"
format_stamp: "Formatted at 2026-09-27 18:40:32 on dist-test-slave-bzzj"
server_key: "68830f3fa28a00eacb89a67227e52a31"
server_key_iv: "c9076fb3d4d1cc5ef93011b02eeb0572"
server_key_version: "encryptionkey@0"
I20260927 18:41:14.517697 22953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:14.532974 22953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:14.535812 22953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:14.535969 22953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:14.536564 22953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:14.537380 22953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:14.537425 22953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.537465 22953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:14.537485 22953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:14.545825 22953 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:36339
I20260927 18:41:14.545849 23081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:36339 every 8 connection(s)
I20260927 18:41:14.546790 22953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:14.547878 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 22953
I20260927 18:41:14.550624 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.549015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:42031)
0927 18:41:14.549446 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.549936 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
W20260927 18:41:14.551067 23082 heartbeater.cc:644] Failed to heartbeat to 127.6.4.62:44179 (0 consecutive failures): Network error: Failed to ping master at 127.6.4.62:44179: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
I20260927 18:41:14.551622 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.551421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:46387)
0927 18:41:14.551525 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.551569 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:41:14.552203 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:14.551990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:53691)
0927 18:41:14.552081 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:14.552155 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 18:41:14.552398 23082 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:41:15.201247 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.200828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:45985)
0927 18:41:15.201081 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:15.201140 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
I20260927 18:41:15.374962 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.374534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:36403)
0927 18:41:15.374793 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:15.374849 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260927 18:41:15.553442 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.552965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:39385)
0927 18:41:15.553238 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:15.553306 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260927 18:41:15.731130 23089 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:15.731403 23089 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:15.732908 23089 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
Sep 27 18:41:15 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.3@KRBTEST.COM
Sep 27 18:41:15 dist-test-slave-bzzj krb5kdc[21509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.2@KRBTEST.COM
I20260927 18:41:15.748066 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.733194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41185 (local address 127.6.4.1:34063)
0927 18:41:15.733912 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:15.733927 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:15.734057 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:15.735764 (+ 1707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:15.735771 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:15.736431 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:15.736951 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:15.736968 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.738294 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.738297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:15.739283 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:15.739293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.739996 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.740018 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:15.740303 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:15.740985 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:15.741004 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:15.743078 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:15.745071 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:15.745074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:15.745075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:15.745423 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:15.745846 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:15.745849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:15.745851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:15.745975 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:15.746813 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:15.746821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:15.747200 (+ 379us) client_negotiation.cc:769] Sending connection context
0927 18:41:15.747445 (+ 245us) client_negotiation.cc:240] Negotiation successful
0927 18:41:15.747904 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":578,"spinlock_wait_cycles":20224,"thread_start_us":61,"threads_started":1}
I20260927 18:41:15.748126 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.733378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:34063 (local address 127.6.4.2:41185)
0927 18:41:15.733965 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:15.733989 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:15.734047 (+ 58us) server_negotiation.cc:407] Connection header received
0927 18:41:15.734965 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:15.734973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:15.735354 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:15.735688 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:15.737126 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.738144 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.739400 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.739892 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.740057 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:15.743276 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:15.743282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:15.743295 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:15.743327 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:15.744944 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:15.745556 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:15.745562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:15.745568 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:15.745736 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:15.746101 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:15.746104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:15.746106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:15.746686 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:15.746834 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:15.747584 (+ 750us) server_negotiation.cc:299] Negotiation successful
0927 18:41:15.747901 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":157,"threads_started":1}
I20260927 18:41:15.748066 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.733194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:36339 (local address 127.6.4.1:56601)
0927 18:41:15.733587 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:15.733607 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:15.733739 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:15.735764 (+ 2025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:15.735771 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:15.736431 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:15.736950 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:15.736968 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.738284 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.738287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:15.739284 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:15.739293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.739998 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.740018 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:15.740304 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:15.741030 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:15.741041 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:15.742725 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:15.745009 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:15.745016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:15.745033 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:15.745431 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:15.745803 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:15.745806 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:15.745808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:15.745974 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:15.746813 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:15.746821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:15.747200 (+ 379us) client_negotiation.cc:769] Sending connection context
0927 18:41:15.747445 (+ 245us) client_negotiation.cc:240] Negotiation successful
0927 18:41:15.747843 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":177,"spinlock_wait_cycles":20480,"thread_start_us":84,"threads_started":1}
I20260927 18:41:15.748126 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:15.733485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:56601 (local address 127.6.4.3:36339)
0927 18:41:15.733811 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:15.733842 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:15.733894 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:41:15.734811 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:15.734820 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:15.735356 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:15.735688 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:15.737122 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.738143 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.739402 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:15.739895 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:15.740053 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:15.742986 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:15.742994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:15.743049 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:15.743084 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:15.744870 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:15.745563 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:15.745569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:15.745575 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:15.745712 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:15.746075 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:15.746078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:15.746080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:15.746686 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:15.746834 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:15.747584 (+ 750us) server_negotiation.cc:299] Negotiation successful
0927 18:41:15.747901 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
W20260927 18:41:15.751132 22703 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
W20260927 18:41:15.751331 22704 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:41:15.751408 22704 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb5df9393f1a4af6b3f4913dcd8080fe; no voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc
I20260927 18:41:15.751674 23089 raft_consensus.cc:2726] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:16.202353 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:16.202067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:44925)
0927 18:41:16.202203 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:16.202267 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:41:16.376154 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:16.375709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:54917)
0927 18:41:16.375970 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:16.376024 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260927 18:41:16.554601 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:16.554189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:34541)
0927 18:41:16.554453 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:16.554512 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":70,"threads_started":1}
I20260927 18:41:17.203610 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:17.203135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:40133)
0927 18:41:17.203439 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:17.203511 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
I20260927 18:41:17.377429 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:17.376989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:51095)
0927 18:41:17.377241 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:17.377305 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260927 18:41:17.543118 23098 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:17.543251 23098 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:17.543640 23098 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
W20260927 18:41:17.544416 22704 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
W20260927 18:41:17.544528 22703 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:41:17.544615 22703 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb5df9393f1a4af6b3f4913dcd8080fe; no voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc
I20260927 18:41:17.544773 23098 raft_consensus.cc:2726] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:17.555819 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:17.555380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:38809)
0927 18:41:17.555650 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:17.555709 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260927 18:41:18.204824 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:18.204386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:60631)
0927 18:41:18.204636 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:18.204690 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260927 18:41:18.387285 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:18.380075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:52705)
0927 18:41:18.387119 (+ 7044us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:18.387184 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6955,"thread_start_us":82,"threads_started":1}
I20260927 18:41:18.557032 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:18.556602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:41077)
0927 18:41:18.556863 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:18.556931 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260927 18:41:19.206287 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:19.205836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:46901)
0927 18:41:19.206126 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:19.206191 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260927 18:41:19.388592 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:19.388155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:33669)
0927 18:41:19.388430 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:19.388493 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":91,"threads_started":1}
I20260927 18:41:19.400861 23105 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:19.401029 23105 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:19.401432 23105 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
W20260927 18:41:19.402151 22704 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
W20260927 18:41:19.402261 22703 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:41:19.402344 22703 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb5df9393f1a4af6b3f4913dcd8080fe; no voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc
I20260927 18:41:19.402486 23105 raft_consensus.cc:2726] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:19.558303 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:19.557862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:34595)
0927 18:41:19.558126 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:19.558190 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260927 18:41:20.207434 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:20.207010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:47883)
0927 18:41:20.207282 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:20.207340 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260927 18:41:20.389789 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:20.389350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:33933)
0927 18:41:20.389624 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:20.389690 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":71,"threads_started":1}
I20260927 18:41:20.559511 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:20.559094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:57103)
0927 18:41:20.559351 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:20.559410 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260927 18:41:21.208643 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:21.208225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:47527)
0927 18:41:21.208488 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:21.208547 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260927 18:41:21.391063 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:21.390576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:51315)
0927 18:41:21.390898 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:21.390964 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":75,"threads_started":1}
I20260927 18:41:21.560752 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:21.560297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:39349)
0927 18:41:21.560599 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:21.560658 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
I20260927 18:41:22.209887 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:22.209455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:49247)
0927 18:41:22.209728 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:22.209790 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":88,"threads_started":1}
I20260927 18:41:22.392234 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:22.391775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:41325)
0927 18:41:22.392049 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:22.392134 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":88,"threads_started":1}
I20260927 18:41:22.561942 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:22.561521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:40971)
0927 18:41:22.561778 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:22.561843 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260927 18:41:22.774564 23116 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:22.774713 23116 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:22.775142 23116 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
W20260927 18:41:22.775926 22704 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
W20260927 18:41:22.776049 22703 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:41:22.776145 22703 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb5df9393f1a4af6b3f4913dcd8080fe; no voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc
I20260927 18:41:22.776288 23116 raft_consensus.cc:2726] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:23.211194 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:23.210729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:50591)
0927 18:41:23.211032 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:23.211099 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":88,"threads_started":1}
I20260927 18:41:23.393384 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:23.392951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:44605)
0927 18:41:23.393216 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:23.393282 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260927 18:41:23.563172 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:23.562722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:60941)
0927 18:41:23.563012 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:23.563075 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":115,"threads_started":1}
I20260927 18:41:24.212683 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:24.212227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:34127)
0927 18:41:24.212530 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:24.212591 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":77,"threads_started":1}
I20260927 18:41:24.394604 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:24.394163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:47333)
0927 18:41:24.394444 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:24.394505 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":73,"threads_started":1}
I20260927 18:41:24.564447 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:24.564020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:45459)
0927 18:41:24.564290 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:24.564351 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260927 18:41:25.213809 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:25.213415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:39515)
0927 18:41:25.213650 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:25.213702 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
I20260927 18:41:25.395853 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:25.395390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:47827)
0927 18:41:25.395688 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:25.395755 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":81,"threads_started":1}
I20260927 18:41:25.565536 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:25.565153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:56179)
0927 18:41:25.565383 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:25.565440 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260927 18:41:26.215021 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:26.214610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:34751)
0927 18:41:26.214862 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:26.214920 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
I20260927 18:41:26.396979 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:26.396575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:52037)
0927 18:41:26.396821 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:26.396882 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260927 18:41:26.566761 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:26.566327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:59197)
0927 18:41:26.566607 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:26.566668 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260927 18:41:27.216396 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:27.215860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:43577)
0927 18:41:27.216209 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:27.216289 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260927 18:41:27.398124 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:27.397689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:51033)
0927 18:41:27.397984 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:27.398034 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
I20260927 18:41:27.567992 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:27.567576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:52267)
0927 18:41:27.567816 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:27.567877 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":83,"threads_started":1}
I20260927 18:41:28.217607 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:28.217180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:46587)
0927 18:41:28.217424 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:28.217487 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
I20260927 18:41:28.399250 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:28.398813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:36115)
0927 18:41:28.399086 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:28.399148 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
I20260927 18:41:28.407457 23134 raft_consensus.cc:482] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:28.407583 23134 raft_consensus.cc:504] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd862bbed3a46ad836a7dc8bb0aad6a" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41185 } } peers { permanent_uuid: "3507259df9be4a908c33406f7a8642cc" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 36339 } } peers { permanent_uuid: "fb5df9393f1a4af6b3f4913dcd8080fe" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 40859 } }
I20260927 18:41:28.407959 23134 leader_election.cc:288] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185), 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339)
W20260927 18:41:28.408702 22704 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bd862bbed3a46ad836a7dc8bb0aad6a (127.6.4.2:41185): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
W20260927 18:41:28.408828 22703 leader_election.cc:341] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3507259df9be4a908c33406f7a8642cc (127.6.4.3:36339): Not found: Tablet not found: fa8b29d6c5b8495c9b89b04eeaebdf79
I20260927 18:41:28.408893 22703 leader_election.cc:302] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb5df9393f1a4af6b3f4913dcd8080fe; no voters: 0bd862bbed3a46ad836a7dc8bb0aad6a, 3507259df9be4a908c33406f7a8642cc
I20260927 18:41:28.409034 23134 raft_consensus.cc:2726] T fa8b29d6c5b8495c9b89b04eeaebdf79 P fb5df9393f1a4af6b3f4913dcd8080fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:41:28.569060 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:28.568665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.3:37893)
0927 18:41:28.568908 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:28.568965 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260927 18:41:29.219002 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:29.218544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.1:35379)
0927 18:41:29.218809 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:29.218868 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260927 18:41:29.400584 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:29.400184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:44179 (local address 127.6.4.2:33451)
0927 18:41:29.400430 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:29.400487 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.62:44179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260927 18:41:29.549033 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22653
I20260927 18:41:29.554132 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22819
I20260927 18:41:29.558740 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 22953
2026-09-27T18:41:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59226 ms)
[----------] 30 tests from SecurityITest (232012 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:29 dist-test-slave-bzzj krb5kdc[23140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:29 dist-test-slave-bzzj krb5kdc[23140](info): set up 2 sockets
Sep 27 18:41:29 dist-test-slave-bzzj krb5kdc[23140](info): commencing operation
krb5kdc: starting...
W20260927 18:41:30.507711 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:30 dist-test-slave-bzzj krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:41:30.621393 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45593
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:42465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:45593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:30.728071 23153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:30.728356 23153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:30.728396 23153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:30.732362 23153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:30.732417 23153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:30.732435 23153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:30.732450 23153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:30.732465 23153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:30.737717 23153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:42465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:45593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:30.738740 23153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:30.739686 23153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:30.748080 23159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:30.748080 23161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:30.748226 23153 server_base.cc:1034] running on GCE node
W20260927 18:41:30.748181 23158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:30.749068 23153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:30.750155 23153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:30.751325 23153 hybrid_clock.cc:648] HybridClock initialized: now 1790534490751293 us; error 42 us; skew 500 ppm
Sep 27 18:41:30 dist-test-slave-bzzj krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:30.753980 23153 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:30.754812 23153 webserver.cc:460] Webserver started at http://127.6.4.62:43827/ using document root <none> and password file <none>
I20260927 18:41:30.755348 23153 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:30.755414 23153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:30.755594 23153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:30.757604 23153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "a9c5307914e949799938785c23217bfa"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-bzzj"
server_key: "457e8a32a3aa917e4e5ec06b933b0c42"
server_key_iv: "97c833f82cd3aaeb842d0eb44fb8957d"
server_key_version: "encryptionkey@0"
I20260927 18:41:30.758123 23153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "a9c5307914e949799938785c23217bfa"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-bzzj"
server_key: "457e8a32a3aa917e4e5ec06b933b0c42"
server_key_iv: "97c833f82cd3aaeb842d0eb44fb8957d"
server_key_version: "encryptionkey@0"
I20260927 18:41:30.761677 23153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260927 18:41:30.763975 23168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:30.765183 23153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:41:30.765292 23153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "a9c5307914e949799938785c23217bfa"
format_stamp: "Formatted at 2026-09-27 18:41:30 on dist-test-slave-bzzj"
server_key: "457e8a32a3aa917e4e5ec06b933b0c42"
server_key_iv: "97c833f82cd3aaeb842d0eb44fb8957d"
server_key_version: "encryptionkey@0"
I20260927 18:41:30.765386 23153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:30.811188 23153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:30.814101 23153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:30.814280 23153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:30.820740 23153 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:45593
I20260927 18:41:30.820761 23220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:45593 every 8 connection(s)
I20260927 18:41:30.821658 23153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:30.824358 23221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:30.828177 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23153
I20260927 18:41:30.828286 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:30.828563 6160 external_mini_cluster.cc:1428] Setting key 6f54a0188980bb546474ea41b9112668
I20260927 18:41:30.829890 23221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa: Bootstrap starting.
I20260927 18:41:30.832542 23221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:30.833400 23221 log.cc:824] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:30.835300 23221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa: No bootstrap required, opened a new log
Sep 27 18:41:30 dist-test-slave-bzzj krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:30.839186 23221 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c5307914e949799938785c23217bfa" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45593 } }
I20260927 18:41:30.839450 23221 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:30.839505 23221 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c5307914e949799938785c23217bfa, State: Initialized, Role: FOLLOWER
I20260927 18:41:30.839975 23221 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c5307914e949799938785c23217bfa" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45593 } }
I20260927 18:41:30.840129 23221 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:30.840200 23221 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:30.840299 23221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:30.841334 23221 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c5307914e949799938785c23217bfa" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45593 } }
I20260927 18:41:30.841677 23221 leader_election.cc:302] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c5307914e949799938785c23217bfa; no voters:
I20260927 18:41:30.842002 23221 leader_election.cc:288] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:30.842090 23226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:30.842309 23226 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [term 1 LEADER]: Becoming Leader. State: Replica: a9c5307914e949799938785c23217bfa, State: Running, Role: LEADER
I20260927 18:41:30.842628 23226 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c5307914e949799938785c23217bfa" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45593 } }
I20260927 18:41:30.842782 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:30.829906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.6.4.62:45593)
0927 18:41:30.830313 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:30.830332 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:30.830384 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:41:30.831190 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:30.831215 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:30.831635 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:30.831976 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:30.832655 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:30.834017 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:30.834828 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:30.835502 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:30.835656 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:30.837946 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:30.837953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:30.837970 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:30.838001 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:30.839762 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:30.840178 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:30.840184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:30.840189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:30.840324 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:30.840593 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:30.840596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:30.840598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:30.841088 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:30.841271 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:30.841527 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 18:41:30.841816 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260927 18:41:30.843254 23221 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:30.843744 23226 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c5307914e949799938785c23217bfa. Latest consensus state: current_term: 1 leader_uuid: "a9c5307914e949799938785c23217bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c5307914e949799938785c23217bfa" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45593 } } }
I20260927 18:41:30.843933 23226 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:30.844159 23227 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c5307914e949799938785c23217bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c5307914e949799938785c23217bfa" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45593 } } }
I20260927 18:41:30.844249 23227 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:30.845364 23233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:30.847988 23233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:30.852855 23233 catalog_manager.cc:1347] Generated new cluster ID: c7d78fc9ee44433d989f126521df4eab
I20260927 18:41:30.852921 23233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:30.865188 23233 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:30.866146 23233 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:30.870602 23233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa: Generated new TSK 0
I20260927 18:41:30.871248 23233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:41:30.903113 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45593
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:42465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:31.013123 23247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.013391 23247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.013451 23247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.017470 23247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.017532 23247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.017613 23247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:31.022816 23247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:42465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:31.023882 23247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.024920 23247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:41:31.034267 23247 server_base.cc:1034] running on GCE node
W20260927 18:41:31.034420 23252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.034396 23253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.034484 23255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.034947 23247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.035543 23247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.036746 23247 hybrid_clock.cc:648] HybridClock initialized: now 1790534491036719 us; error 42 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-bzzj krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.039417 23247 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:41:31.040313 23247 webserver.cc:460] Webserver started at http://127.6.4.1:40863/ using document root <none> and password file <none>
I20260927 18:41:31.040848 23247 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.040912 23247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:31.041090 23247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:31.043063 23247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "3637e4fdcd264f4ab98a93e7601b75e4"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "230d0f6dd30b948bb464316042573062"
server_key_iv: "491e092efce3c1d35526658b83d9ae1d"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.043618 23247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "3637e4fdcd264f4ab98a93e7601b75e4"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "230d0f6dd30b948bb464316042573062"
server_key_iv: "491e092efce3c1d35526658b83d9ae1d"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.047241 23247 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260927 18:41:31.049638 23262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.050781 23247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 18:41:31.050894 23247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "3637e4fdcd264f4ab98a93e7601b75e4"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "230d0f6dd30b948bb464316042573062"
server_key_iv: "491e092efce3c1d35526658b83d9ae1d"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.051019 23247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.078642 23247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.081614 23247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.081766 23247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.082293 23247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:31.083106 23247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:31.083153 23247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.083204 23247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:31.083232 23247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.091180 23247 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:39055
I20260927 18:41:31.091233 23375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:39055 every 8 connection(s)
I20260927 18:41:31.092032 23247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:31.099653 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23247
I20260927 18:41:31.099792 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:31.100132 6160 external_mini_cluster.cc:1428] Setting key 09272547f921bea19e4e1b4a687d1a48
Sep 27 18:41:31 dist-test-slave-bzzj krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:31.105720 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.093780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:37557 (local address 127.6.4.62:45593)
0927 18:41:31.093922 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.093927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.094764 (+ 837us) server_negotiation.cc:407] Connection header received
0927 18:41:31.096079 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.096082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.096168 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.096244 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.097405 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.097972 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.099091 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.099308 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.099382 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.102027 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.102030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.102033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:31.102058 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.103459 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.104072 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.104075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.104076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.104192 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.104582 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.104585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.104586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.104780 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.104858 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.105450 (+ 592us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.105559 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:41:31.106835 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.094184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45593 (local address 127.6.4.1:37557)
0927 18:41:31.094619 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.094662 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.095823 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.096398 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.096405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.096751 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:31.097267 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.097281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.098094 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.098097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.098992 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.099001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.099407 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.099464 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.099651 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.100310 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:31.100326 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:31.101870 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:31.103592 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.103599 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.103616 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.103957 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.104284 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.104287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.104289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.104472 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.104878 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:31.104884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:31.105190 (+ 306us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.105410 (+ 220us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.105760 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260927 18:41:31.108049 23376 heartbeater.cc:347] Connected to a master server at 127.6.4.62:45593
I20260927 18:41:31.108297 23376 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:31.108752 23376 heartbeater.cc:511] Master 127.6.4.62:45593 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:41:31.110337 23185 ts_manager.cc:194] Registered new tserver with Master: 3637e4fdcd264f4ab98a93e7601b75e4 (127.6.4.1:39055)
I20260927 18:41:31.111487 23185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:37557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:41:31.126520 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45593
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:42465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:31.234102 23381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.234376 23381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.234467 23381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.238466 23381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.238534 23381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.238617 23381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:31.243959 23381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:42465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:31.245070 23381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.246085 23381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.255474 23386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.255497 23389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.255525 23381 server_base.cc:1034] running on GCE node
W20260927 18:41:31.255475 23387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.256060 23381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.256733 23381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.257910 23381 hybrid_clock.cc:648] HybridClock initialized: now 1790534491257891 us; error 30 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-bzzj krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.264793 23381 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:41:31.265681 23381 webserver.cc:460] Webserver started at http://127.6.4.2:35681/ using document root <none> and password file <none>
I20260927 18:41:31.266219 23381 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.266281 23381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:31.266455 23381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:31.268581 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "c4470f5fd82a4982bc0f2efd6f17faeb"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "fbc7efea7afec7df6880d6d503a7cda7"
server_key_iv: "83f3cadb94215a6cf64982d8f113859f"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.269163 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "c4470f5fd82a4982bc0f2efd6f17faeb"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "fbc7efea7afec7df6880d6d503a7cda7"
server_key_iv: "83f3cadb94215a6cf64982d8f113859f"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.272914 23381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 18:41:31.275298 23396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.276535 23381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:31.276633 23381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "c4470f5fd82a4982bc0f2efd6f17faeb"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "fbc7efea7afec7df6880d6d503a7cda7"
server_key_iv: "83f3cadb94215a6cf64982d8f113859f"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.276738 23381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.308207 23381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.311152 23381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.311331 23381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.311916 23381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:31.312778 23381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:31.312831 23381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.312875 23381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:31.312904 23381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.321142 23381 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:40357
I20260927 18:41:31.321173 23509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:40357 every 8 connection(s)
I20260927 18:41:31.322046 23381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:31.322678 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23381
I20260927 18:41:31.322817 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:31.323107 6160 external_mini_cluster.cc:1428] Setting key d1edc5c050d4edf542aafcff298de78d
Sep 27 18:41:31 dist-test-slave-bzzj krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:41:31.335963 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.323858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:34313 (local address 127.6.4.62:45593)
0927 18:41:31.323975 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.323979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.325002 (+ 1023us) server_negotiation.cc:407] Connection header received
0927 18:41:31.326634 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.326637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.326689 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.326786 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.328067 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.328639 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.329777 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.330127 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.330245 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.332699 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.332703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.332705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:31.332733 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.333971 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.334476 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.334479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.334480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.334562 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.334861 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.334864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.334866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.335088 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.335186 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.335736 (+ 550us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.335839 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:41:31.337018 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.324315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45593 (local address 127.6.4.2:34313)
0927 18:41:31.324840 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.324895 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.326413 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.326925 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.326933 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.327289 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:31.327910 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.327930 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.328776 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.328779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.329652 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.329661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.330225 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.330270 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.330410 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.330985 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:31.331001 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:31.332525 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:31.334069 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.334074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.334086 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.334396 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.334643 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.334645 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.334646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.334780 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.335185 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:31.335191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:31.335508 (+ 317us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.335707 (+ 199us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.336042 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":148,"threads_started":1}
I20260927 18:41:31.338209 23510 heartbeater.cc:347] Connected to a master server at 127.6.4.62:45593
I20260927 18:41:31.338445 23510 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:31.338969 23510 heartbeater.cc:511] Master 127.6.4.62:45593 requested a full tablet report, sending...
I20260927 18:41:31.339989 23185 ts_manager.cc:194] Registered new tserver with Master: c4470f5fd82a4982bc0f2efd6f17faeb (127.6.4.2:40357)
I20260927 18:41:31.340565 23185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:34313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:41:31.349007 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45593
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:42465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:31.456902 23515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.457158 23515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.457201 23515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.461143 23515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.461201 23515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.461259 23515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:31.466473 23515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:42465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:45593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:31.467463 23515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.468468 23515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.477619 23523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.477619 23520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.477695 23521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.477953 23515 server_base.cc:1034] running on GCE node
I20260927 18:41:31.478304 23515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.478892 23515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.480032 23515 hybrid_clock.cc:648] HybridClock initialized: now 1790534491480006 us; error 38 us; skew 500 ppm
Sep 27 18:41:31 dist-test-slave-bzzj krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:31.482621 23515 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:41:31.483464 23515 webserver.cc:460] Webserver started at http://127.6.4.3:32801/ using document root <none> and password file <none>
I20260927 18:41:31.483994 23515 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.484052 23515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:31.484264 23515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:31.486243 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "f891e039055d45ec8960b6e0cbc60d6b"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "5862ba8235e68879b6e1d325a400a789"
server_key_iv: "8d3434b501c92b9069278432017439a1"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.486744 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "f891e039055d45ec8960b6e0cbc60d6b"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "5862ba8235e68879b6e1d325a400a789"
server_key_iv: "8d3434b501c92b9069278432017439a1"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.490244 23515 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260927 18:41:31.492515 23530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.493714 23515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:31.493865 23515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "f891e039055d45ec8960b6e0cbc60d6b"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "5862ba8235e68879b6e1d325a400a789"
server_key_iv: "8d3434b501c92b9069278432017439a1"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.493971 23515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.514076 23515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.516897 23515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.517076 23515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.517645 23515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:31.518450 23515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:31.518502 23515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.518555 23515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:31.518582 23515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.526417 23515 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:45955
I20260927 18:41:31.526439 23643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:45955 every 8 connection(s)
I20260927 18:41:31.527264 23515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:31.534785 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23515
I20260927 18:41:31.534910 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:31.535205 6160 external_mini_cluster.cc:1428] Setting key 724890a81fcca2539ccbf90f8e2a8da3
Sep 27 18:41:31 dist-test-slave-bzzj krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534491, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:31.539708 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.529009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:49977 (local address 127.6.4.62:45593)
0927 18:41:31.529170 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.529175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.529920 (+ 745us) server_negotiation.cc:407] Connection header received
0927 18:41:31.531109 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.531113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.531159 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.531228 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.532353 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.532871 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.533938 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.534138 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.534203 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.536744 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.536749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.536751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:31.536774 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.538099 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.538512 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.538514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.538515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.538582 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.538833 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.538835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.538836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.539021 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.539091 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.539495 (+ 404us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.539589 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:41:31.540540 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.529357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:45593 (local address 127.6.4.3:49977)
0927 18:41:31.529774 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.529815 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.530927 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.531386 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.531393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.531729 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:31.532244 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.532259 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.532955 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.532957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.533850 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.533858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.534201 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.534245 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.534389 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.534993 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:31.535010 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:31.536604 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:31.538191 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.538195 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.538207 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.538438 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.538645 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:31.538647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:31.538648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:31.538768 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:31.539094 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:31.539098 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:31.539318 (+ 220us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.539466 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.539709 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":111,"threads_started":1}
I20260927 18:41:31.541527 23644 heartbeater.cc:347] Connected to a master server at 127.6.4.62:45593
I20260927 18:41:31.541720 23644 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:31.542142 23644 heartbeater.cc:511] Master 127.6.4.62:45593 requested a full tablet report, sending...
I20260927 18:41:31.543071 23185 ts_manager.cc:194] Registered new tserver with Master: f891e039055d45ec8960b6e0cbc60d6b (127.6.4.3:45955)
I20260927 18:41:31.543607 23185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:49977
I20260927 18:41:31.548760 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:31.557119 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.550564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.6.4.62:45593)
0927 18:41:31.550693 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.550698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.550829 (+ 131us) server_negotiation.cc:407] Connection header received
0927 18:41:31.550931 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.550933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.550971 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.551030 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.551597 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.552086 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.552856 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.553062 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.553125 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.554283 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.554286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.554288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:31.554309 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.555517 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.555936 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.555939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.555940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.556020 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:31.556271 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:31.556274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:31.556276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:31.556529 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.556638 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.556836 (+ 198us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.556945 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:41:31.560320 23185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:31.563814 23185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:31.579754 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.573683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.6.4.3:45955)
0927 18:41:31.574571 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.574584 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.574612 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:41:31.574802 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.574809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.575115 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.575289 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:31.575640 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.576799 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.578188 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.578968 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.579080 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.579261 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.579434 (+ 173us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.579561 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":177,"threads_started":1}
I20260927 18:41:31.580361 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.574869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.6.4.2:40357)
0927 18:41:31.575316 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.575327 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.575349 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:41:31.575425 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.575429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.575604 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.575734 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:31.576190 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.577269 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.578689 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.579417 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.579517 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.579670 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.580079 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.580206 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":150,"threads_started":1}
I20260927 18:41:31.580518 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.573564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:45955 (local address 127.0.0.1:50170)
0927 18:41:31.574353 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.574439 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.574579 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.575314 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.575320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.575341 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:31.575509 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.575517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.576964 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.576968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.578056 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.578066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.579090 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.579094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.579211 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.579256 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.579379 (+ 123us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.579570 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":122,"thread_start_us":81,"threads_started":1}
I20260927 18:41:31.580513 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.574558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.6.4.1:39055)
0927 18:41:31.575029 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.575044 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.575070 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:41:31.575172 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.575178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.575386 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.575560 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:31.575727 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.576738 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.578507 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.579337 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.579965 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.580189 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.580240 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.580340 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":143,"threads_started":1}
I20260927 18:41:31.580988 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.574403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:40357 (local address 127.0.0.1:50216)
0927 18:41:31.575103 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.575115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.575193 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.575909 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.575912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.575931 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:31.576045 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.576051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.577417 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.577421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.578585 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.578592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.579729 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.579733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.579823 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.579848 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.579910 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.579984 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":52,"thread_start_us":93,"threads_started":1}
I20260927 18:41:31.580971 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.573883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39055 (local address 127.0.0.1:36602)
0927 18:41:31.574826 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.574839 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.574926 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.575519 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.575522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.575538 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:31.575628 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.575633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.576879 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.576882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.578376 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.578386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.579481 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.579485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.579638 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.579664 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.579715 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.579785 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":54,"thread_start_us":110,"threads_started":1}
I20260927 18:41:31.583932 23444 tablet_service.cc:1467] Processing CreateTablet for tablet 25e246da0b594b488a7d32f763fef025 (DEFAULT_TABLE table=test-table [id=c33b9572bcb840eba61b4252a3d3bbfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:31.583933 23310 tablet_service.cc:1467] Processing CreateTablet for tablet 25e246da0b594b488a7d32f763fef025 (DEFAULT_TABLE table=test-table [id=c33b9572bcb840eba61b4252a3d3bbfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:31.583933 23578 tablet_service.cc:1467] Processing CreateTablet for tablet 25e246da0b594b488a7d32f763fef025 (DEFAULT_TABLE table=test-table [id=c33b9572bcb840eba61b4252a3d3bbfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:31.585235 23578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e246da0b594b488a7d32f763fef025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:31.585237 23444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e246da0b594b488a7d32f763fef025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:31.585243 23310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e246da0b594b488a7d32f763fef025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:31.592321 23661 tablet_bootstrap.cc:492] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Bootstrap starting.
I20260927 18:41:31.593080 23662 tablet_bootstrap.cc:492] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: Bootstrap starting.
I20260927 18:41:31.593418 23663 tablet_bootstrap.cc:492] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4: Bootstrap starting.
I20260927 18:41:31.594782 23661 tablet_bootstrap.cc:654] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:31.595214 23663 tablet_bootstrap.cc:654] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:31.595853 23661 log.cc:824] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:31.596197 23663 log.cc:824] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:31.597677 23661 tablet_bootstrap.cc:492] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: No bootstrap required, opened a new log
I20260927 18:41:31.597863 23661 ts_tablet_manager.cc:1397] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260927 18:41:31.598068 23662 tablet_bootstrap.cc:654] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:31.598105 23663 tablet_bootstrap.cc:492] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4: No bootstrap required, opened a new log
I20260927 18:41:31.598265 23663 ts_tablet_manager.cc:1397] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:41:31.599231 23662 log.cc:824] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:31.601192 23662 tablet_bootstrap.cc:492] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: No bootstrap required, opened a new log
I20260927 18:41:31.600992 23661 raft_consensus.cc:348] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.601261 23661 raft_consensus.cc:374] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:31.601356 23661 raft_consensus.cc:729] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f891e039055d45ec8960b6e0cbc60d6b, State: Initialized, Role: FOLLOWER
I20260927 18:41:31.601774 23662 ts_tablet_manager.cc:1397] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260927 18:41:31.601851 23661 consensus_queue.cc:260] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.601815 23663 raft_consensus.cc:348] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.602097 23663 raft_consensus.cc:374] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:31.602190 23663 raft_consensus.cc:729] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3637e4fdcd264f4ab98a93e7601b75e4, State: Initialized, Role: FOLLOWER
I20260927 18:41:31.602448 23644 heartbeater.cc:503] Master 127.6.4.62:45593 was elected leader, sending a full tablet report...
I20260927 18:41:31.602674 23663 consensus_queue.cc:260] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.602803 23661 ts_tablet_manager.cc:1428] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260927 18:41:31.603467 23376 heartbeater.cc:503] Master 127.6.4.62:45593 was elected leader, sending a full tablet report...
I20260927 18:41:31.603575 23663 ts_tablet_manager.cc:1428] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 18:41:31.606315 23662 raft_consensus.cc:348] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.606642 23662 raft_consensus.cc:374] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:31.606702 23662 raft_consensus.cc:729] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4470f5fd82a4982bc0f2efd6f17faeb, State: Initialized, Role: FOLLOWER
I20260927 18:41:31.607101 23662 consensus_queue.cc:260] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.607661 23510 heartbeater.cc:503] Master 127.6.4.62:45593 was elected leader, sending a full tablet report...
I20260927 18:41:31.607956 23662 ts_tablet_manager.cc:1428] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:31.639140 23667 raft_consensus.cc:482] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:31.639360 23667 raft_consensus.cc:504] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.640600 23667 leader_election.cc:288] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4470f5fd82a4982bc0f2efd6f17faeb (127.6.4.2:40357), 3637e4fdcd264f4ab98a93e7601b75e4 (127.6.4.1:39055)
I20260927 18:41:31.645190 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.640913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:53841 (local address 127.6.4.2:40357)
0927 18:41:31.641023 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.641028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.641110 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:41:31.641232 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.641234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.641278 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.641356 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:31.641594 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.642337 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.643921 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.644592 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.644698 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.644761 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.644991 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.645061 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:41:31.645190 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.640938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:37343 (local address 127.6.4.1:39055)
0927 18:41:31.641059 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.641063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.641324 (+ 261us) server_negotiation.cc:407] Connection header received
0927 18:41:31.641408 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.641411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.641456 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.641515 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:31.641733 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.642454 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.643970 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.644623 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.644709 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.644761 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.644991 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.645061 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:41:31.645205 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.640818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:40357 (local address 127.6.4.3:53841)
0927 18:41:31.641021 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.641036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.641131 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.641377 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.641380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.641393 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:31.641494 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.641499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.642449 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.642451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.643800 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.643808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.644703 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.644706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.644832 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.644872 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.644941 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.645067 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":77}
I20260927 18:41:31.645217 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.640841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39055 (local address 127.6.4.3:37343)
0927 18:41:31.641235 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.641245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.641327 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.641540 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:31.641543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:31.641559 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:31.641649 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.641654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.642537 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.642541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.643797 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:31.643808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.644713 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.644717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:31.644822 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.644867 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 18:41:31.644941 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:41:31.645053 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":35,"thread_start_us":130,"threads_started":1}
I20260927 18:41:31.645897 23464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e246da0b594b488a7d32f763fef025" candidate_uuid: "f891e039055d45ec8960b6e0cbc60d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" is_pre_election: true
I20260927 18:41:31.645921 23330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e246da0b594b488a7d32f763fef025" candidate_uuid: "f891e039055d45ec8960b6e0cbc60d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" is_pre_election: true
I20260927 18:41:31.646178 23464 raft_consensus.cc:2454] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f891e039055d45ec8960b6e0cbc60d6b in term 0.
I20260927 18:41:31.646181 23330 raft_consensus.cc:2454] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f891e039055d45ec8960b6e0cbc60d6b in term 0.
I20260927 18:41:31.646641 23531 leader_election.cc:302] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3637e4fdcd264f4ab98a93e7601b75e4, c4470f5fd82a4982bc0f2efd6f17faeb, f891e039055d45ec8960b6e0cbc60d6b; no voters:
I20260927 18:41:31.646924 23667 raft_consensus.cc:2781] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:31.646998 23667 raft_consensus.cc:482] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:31.647047 23667 raft_consensus.cc:3037] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:31.648022 23667 raft_consensus.cc:504] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.648402 23667 leader_election.cc:288] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [CANDIDATE]: Term 1 election: Requested vote from peers c4470f5fd82a4982bc0f2efd6f17faeb (127.6.4.2:40357), 3637e4fdcd264f4ab98a93e7601b75e4 (127.6.4.1:39055)
I20260927 18:41:31.648682 23464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e246da0b594b488a7d32f763fef025" candidate_uuid: "f891e039055d45ec8960b6e0cbc60d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb"
I20260927 18:41:31.648739 23330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e246da0b594b488a7d32f763fef025" candidate_uuid: "f891e039055d45ec8960b6e0cbc60d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3637e4fdcd264f4ab98a93e7601b75e4"
I20260927 18:41:31.648801 23464 raft_consensus.cc:3037] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:31.648856 23330 raft_consensus.cc:3037] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:31.649981 23330 raft_consensus.cc:2454] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f891e039055d45ec8960b6e0cbc60d6b in term 1.
I20260927 18:41:31.649981 23464 raft_consensus.cc:2454] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f891e039055d45ec8960b6e0cbc60d6b in term 1.
I20260927 18:41:31.650312 23533 leader_election.cc:302] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3637e4fdcd264f4ab98a93e7601b75e4, c4470f5fd82a4982bc0f2efd6f17faeb, f891e039055d45ec8960b6e0cbc60d6b; no voters:
I20260927 18:41:31.650477 23667 raft_consensus.cc:2781] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:31.650701 23667 raft_consensus.cc:686] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 1 LEADER]: Becoming Leader. State: Replica: f891e039055d45ec8960b6e0cbc60d6b, State: Running, Role: LEADER
I20260927 18:41:31.651000 23667 consensus_queue.cc:237] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } }
I20260927 18:41:31.654171 23185 catalog_manager.cc:5495] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b reported cstate change: term changed from 0 to 1, leader changed from <none> to f891e039055d45ec8960b6e0cbc60d6b (127.6.4.3). New cstate: current_term: 1 leader_uuid: "f891e039055d45ec8960b6e0cbc60d6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f891e039055d45ec8960b6e0cbc60d6b" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 45955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:31.676126 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.672806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.6.4.3:45955)
0927 18:41:31.672922 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.672925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.672938 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:41:31.673014 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.673016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.673069 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.673156 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:31.673384 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.673920 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.674824 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.675033 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.675104 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.675326 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:31.675727 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:31.675832 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.675901 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.675949 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:41:31.681862 23330 raft_consensus.cc:1261] T 25e246da0b594b488a7d32f763fef025 P 3637e4fdcd264f4ab98a93e7601b75e4 [term 1 FOLLOWER]: Refusing update from remote peer f891e039055d45ec8960b6e0cbc60d6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:31.681885 23464 raft_consensus.cc:1261] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 1 FOLLOWER]: Refusing update from remote peer f891e039055d45ec8960b6e0cbc60d6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:31.682480 23667 consensus_queue.cc:1035] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4470f5fd82a4982bc0f2efd6f17faeb" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:31.682657 23671 consensus_queue.cc:1035] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3637e4fdcd264f4ab98a93e7601b75e4" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:31.690153 23675 mvcc.cc:205] Tried to move back new op lower bound from 7334029277921996800 to 7334029277804929024. Current Snapshot: MvccSnapshot[applied={T|T < 7334029277921996800}]
I20260927 18:41:31.691686 23676 mvcc.cc:205] Tried to move back new op lower bound from 7334029277921996800 to 7334029277804929024. Current Snapshot: MvccSnapshot[applied={T|T < 7334029277921996800}]
I20260927 18:41:31.698005 23184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:31.698148 23184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:31.700273 23184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9c5307914e949799938785c23217bfa: Sending DeleteTablet for 3 replicas of tablet 25e246da0b594b488a7d32f763fef025
I20260927 18:41:31.701031 23310 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e246da0b594b488a7d32f763fef025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:31 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:36602
I20260927 18:41:31.701069 23444 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e246da0b594b488a7d32f763fef025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:31 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:50216
I20260927 18:41:31.701231 23578 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e246da0b594b488a7d32f763fef025 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:31 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:50170
I20260927 18:41:31.701525 23684 tablet_replica.cc:330] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: stopping tablet replica
I20260927 18:41:31.701597 23685 tablet_replica.cc:330] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: stopping tablet replica
I20260927 18:41:31.701869 23684 raft_consensus.cc:2229] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:31.701920 23685 raft_consensus.cc:2229] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:31.701957 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23247
I20260927 18:41:31.702760 23684 raft_consensus.cc:2258] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:31.703612 23685 raft_consensus.cc:2258] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 18:41:31.707055 23169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:39055 (error 108)
W20260927 18:41:31.708525 23658 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:41:31.707769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:39055 (local address 127.0.0.1:43526)
0927 18:41:31.707896 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.707913 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:31.707998 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:31.708420 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:39055: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":43}
I20260927 18:41:31.708607 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23381
I20260927 18:41:31.708917 23684 ts_tablet_manager.cc:1905] T 25e246da0b594b488a7d32f763fef025 P c4470f5fd82a4982bc0f2efd6f17faeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:31.709743 23685 ts_tablet_manager.cc:1905] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:41:31.709998 23169 catalog_manager.cc:4584] TS 3637e4fdcd264f4ab98a93e7601b75e4 (127.6.4.1:39055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25e246da0b594b488a7d32f763fef025: Network error: Client connection negotiation failed: client connection to 127.6.4.1:39055: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:41:31.712445 23685 ts_tablet_manager.cc:1918] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:31.712543 23685 log.cc:1196] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/25e246da0b594b488a7d32f763fef025
I20260927 18:41:31.712891 23685 ts_tablet_manager.cc:1939] T 25e246da0b594b488a7d32f763fef025 P f891e039055d45ec8960b6e0cbc60d6b: Deleting consensus metadata
I20260927 18:41:31.714234 23171 catalog_manager.cc:4841] TS f891e039055d45ec8960b6e0cbc60d6b (127.6.4.3:45955): tablet 25e246da0b594b488a7d32f763fef025 (table test-table [id=c33b9572bcb840eba61b4252a3d3bbfa]) successfully deleted
I20260927 18:41:31.714571 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23515
I20260927 18:41:31.715615 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.715316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:40357 (local address 127.0.0.1:34470)
0927 18:41:31.715469 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:31.715535 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.2:40357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260927 18:41:31.719228 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23153
2026-09-27T18:41:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2160 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T18:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:31Z Disabled control of system clock
I20260927 18:41:31.746835 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38297
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:38297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:41:31.854065 23690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:31.854321 23690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:31.854369 23690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:31.858353 23690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:31.858412 23690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:31.858435 23690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:31.858451 23690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:31.858469 23690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:31.863204 23690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:38297
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:31.864281 23690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:31.865667 23690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:31.873919 23695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:31.873881 23696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.874135 23690 server_base.cc:1034] running on GCE node
W20260927 18:41:31.873894 23698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:31.874680 23690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:31.875738 23690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:31.876924 23690 hybrid_clock.cc:648] HybridClock initialized: now 1790534491876900 us; error 33 us; skew 500 ppm
I20260927 18:41:31.878741 23690 webserver.cc:460] Webserver started at http://127.6.4.62:38407/ using document root <none> and password file <none>
I20260927 18:41:31.879303 23690 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:31.879369 23690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:31.879552 23690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:31.881636 23690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "207c46d596fb49968fd4d61482e2911f"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "e9a3fa4552ab1201d59279bd4b80f5e1"
server_key_iv: "5e51d9294f9c0f377ae54b4225778d5f"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.882175 23690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "207c46d596fb49968fd4d61482e2911f"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "e9a3fa4552ab1201d59279bd4b80f5e1"
server_key_iv: "5e51d9294f9c0f377ae54b4225778d5f"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.885810 23690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:31.888077 23704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:31.889278 23690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:31.889400 23690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "207c46d596fb49968fd4d61482e2911f"
format_stamp: "Formatted at 2026-09-27 18:41:31 on dist-test-slave-bzzj"
server_key: "e9a3fa4552ab1201d59279bd4b80f5e1"
server_key_iv: "5e51d9294f9c0f377ae54b4225778d5f"
server_key_version: "encryptionkey@0"
I20260927 18:41:31.889497 23690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:31.904680 23690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:31.905372 23690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:31.905546 23690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:31.912140 23690 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:38297
I20260927 18:41:31.912164 23756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:38297 every 8 connection(s)
I20260927 18:41:31.913050 23690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:31.916028 23757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:31.921257 23757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f: Bootstrap starting.
I20260927 18:41:31.922250 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23690
I20260927 18:41:31.922376 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:31.922648 6160 external_mini_cluster.cc:1428] Setting key c389d06f7881382bffb8539761aadfcb
I20260927 18:41:31.924036 23757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:31.924772 23757 log.cc:824] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:31.926677 23757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f: No bootstrap required, opened a new log
I20260927 18:41:31.930635 23757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207c46d596fb49968fd4d61482e2911f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38297 } }
I20260927 18:41:31.930850 23757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:31.930912 23757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207c46d596fb49968fd4d61482e2911f, State: Initialized, Role: FOLLOWER
I20260927 18:41:31.931303 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:31.923773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.6.4.62:38297)
0927 18:41:31.924410 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:31.924423 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:31.924457 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:41:31.925086 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:31.925104 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:31.925445 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:31.925715 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:31.926035 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.927012 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.928184 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:31.928651 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:31.928770 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:31.929569 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:31.929574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:31.929587 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:31.929961 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:31.930019 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:31.930038 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:31.930109 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:31.930244 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 18:41:31.930515 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260927 18:41:31.931308 23757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207c46d596fb49968fd4d61482e2911f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38297 } }
I20260927 18:41:31.931522 23757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:31.931591 23757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:31.931684 23757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:31.932684 23757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207c46d596fb49968fd4d61482e2911f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38297 } }
I20260927 18:41:31.932986 23757 leader_election.cc:302] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 207c46d596fb49968fd4d61482e2911f; no voters:
I20260927 18:41:31.933255 23757 leader_election.cc:288] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:31.933353 23762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:31.933559 23762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [term 1 LEADER]: Becoming Leader. State: Replica: 207c46d596fb49968fd4d61482e2911f, State: Running, Role: LEADER
I20260927 18:41:31.933887 23762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207c46d596fb49968fd4d61482e2911f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38297 } }
I20260927 18:41:31.934394 23757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:31.934935 23762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207c46d596fb49968fd4d61482e2911f. Latest consensus state: current_term: 1 leader_uuid: "207c46d596fb49968fd4d61482e2911f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207c46d596fb49968fd4d61482e2911f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38297 } } }
I20260927 18:41:31.935194 23762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:31.935285 23763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "207c46d596fb49968fd4d61482e2911f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207c46d596fb49968fd4d61482e2911f" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38297 } } }
I20260927 18:41:31.935429 23763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:31.935665 23771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:31.938277 23771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:31.943579 23771 catalog_manager.cc:1347] Generated new cluster ID: 1bea5f3c1c1f4743b137c4019b1713fe
I20260927 18:41:31.943681 23771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:31.954335 23771 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:31.955336 23771 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:31.960640 23771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f: Generated new TSK 0
I20260927 18:41:31.961306 23771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:31.964085 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38297
--builtin_ntp_servers=127.6.4.20:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:41:32.077379 23781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:32.077668 23781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:32.077708 23781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:32.081931 23781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:32.081985 23781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:32.082041 23781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:32.086802 23781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:32.088172 23781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:32.089538 23781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:32.098888 23789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.098901 23787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.098888 23786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.099104 23781 server_base.cc:1034] running on GCE node
I20260927 18:41:32.099495 23781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:32.100126 23781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:32.101286 23781 hybrid_clock.cc:648] HybridClock initialized: now 1790534492101268 us; error 30 us; skew 500 ppm
I20260927 18:41:32.103063 23781 webserver.cc:460] Webserver started at http://127.6.4.1:45867/ using document root <none> and password file <none>
I20260927 18:41:32.103603 23781 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:32.103664 23781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:32.103854 23781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:32.105922 23781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "77777c4dc30c435aaca0ee59ce98ad02"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "e5cc93fb371e6a8484891347825ddcb8"
server_key_iv: "5bfe3ffc60deeb016a0af3fe0be67225"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.106438 23781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "77777c4dc30c435aaca0ee59ce98ad02"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "e5cc93fb371e6a8484891347825ddcb8"
server_key_iv: "5bfe3ffc60deeb016a0af3fe0be67225"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.109997 23781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:32.112344 23795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.113492 23781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:32.113602 23781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "77777c4dc30c435aaca0ee59ce98ad02"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "e5cc93fb371e6a8484891347825ddcb8"
server_key_iv: "5bfe3ffc60deeb016a0af3fe0be67225"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.113695 23781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:32.131050 23781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:32.131736 23781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:32.131947 23781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:32.132570 23781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:32.133500 23781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:32.133548 23781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.133600 23781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:32.133626 23781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.142158 23781 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:46271
I20260927 18:41:32.142227 23908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:46271 every 8 connection(s)
I20260927 18:41:32.143105 23781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:32.150168 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23781
I20260927 18:41:32.150291 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:32.150627 6160 external_mini_cluster.cc:1428] Setting key cfe6b9d11d3440aeaea3396da877f692
I20260927 18:41:32.152154 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.144949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:43117 (local address 127.6.4.62:38297)
0927 18:41:32.145102 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.145105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.146197 (+ 1092us) server_negotiation.cc:407] Connection header received
0927 18:41:32.147504 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.147507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.147558 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.147632 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.148526 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.149066 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.150178 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.150502 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.150609 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.151677 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.151681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.151683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:32.151695 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.151712 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:32.151719 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.151776 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.151959 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.152014 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 18:41:32.152588 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38297
--builtin_ntp_servers=127.6.4.20:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:41:32.153294 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.145397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:38297 (local address 127.6.4.1:43117)
0927 18:41:32.146042 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.146084 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.147302 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.147784 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.147790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.147873 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:32.148377 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.148388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.149168 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.149170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.150049 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.150057 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.150597 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.150645 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.150803 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.151423 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:32.151425 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:32.151466 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:32.151531 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:32.151788 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:32.151797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:32.151803 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.151949 (+ 146us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.152258 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
I20260927 18:41:32.154594 23909 heartbeater.cc:347] Connected to a master server at 127.6.4.62:38297
I20260927 18:41:32.154820 23909 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.155332 23909 heartbeater.cc:511] Master 127.6.4.62:38297 requested a full tablet report, sending...
I20260927 18:41:32.157015 23721 ts_manager.cc:194] Registered new tserver with Master: 77777c4dc30c435aaca0ee59ce98ad02 (127.6.4.1:46271)
I20260927 18:41:32.158582 23721 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.1:43117
W20260927 18:41:32.262728 23912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:32.263000 23912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:32.263038 23912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:32.266978 23912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:32.267037 23912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:32.267092 23912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:32.271831 23912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:32.272927 23912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:32.274252 23912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:32.283413 23917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.283406 23920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.283468 23912 server_base.cc:1034] running on GCE node
W20260927 18:41:32.283365 23918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.283995 23912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:32.284631 23912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:32.285792 23912 hybrid_clock.cc:648] HybridClock initialized: now 1790534492285756 us; error 54 us; skew 500 ppm
I20260927 18:41:32.287611 23912 webserver.cc:460] Webserver started at http://127.6.4.2:38583/ using document root <none> and password file <none>
I20260927 18:41:32.288201 23912 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:32.288265 23912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:32.288460 23912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:32.290452 23912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "d89d3f01a5a14a6ab14c8ed7577b0423"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "be565398a2b8fca7dc8bcd464675fc8a"
server_key_iv: "b196809ad32618708db4d1b9d694985d"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.290977 23912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "d89d3f01a5a14a6ab14c8ed7577b0423"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "be565398a2b8fca7dc8bcd464675fc8a"
server_key_iv: "b196809ad32618708db4d1b9d694985d"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.294665 23912 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 18:41:32.297272 23926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.298463 23912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:32.298571 23912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "d89d3f01a5a14a6ab14c8ed7577b0423"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "be565398a2b8fca7dc8bcd464675fc8a"
server_key_iv: "b196809ad32618708db4d1b9d694985d"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.298673 23912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:32.320376 23912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:32.321089 23912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:32.321261 23912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:32.321856 23912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:32.322806 23912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:32.322890 23912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.322932 23912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:32.322961 23912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.331261 23912 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:44933
I20260927 18:41:32.331281 24039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:44933 every 8 connection(s)
I20260927 18:41:32.332289 23912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:32.339092 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 23912
I20260927 18:41:32.339211 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:32.339490 6160 external_mini_cluster.cc:1428] Setting key 947c79b28892d68df6a1e76c6c5fd6a0
I20260927 18:41:32.341208 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.334013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:37493 (local address 127.6.4.62:38297)
0927 18:41:32.334185 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.334189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.335279 (+ 1090us) server_negotiation.cc:407] Connection header received
0927 18:41:32.336669 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.336671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.336717 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.336784 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.337620 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.338150 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.339246 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.339564 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.339654 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.340751 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.340755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.340757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:32.340769 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.340790 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:32.340799 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.340856 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.341010 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.341069 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:41:32.341465 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38297
--builtin_ntp_servers=127.6.4.20:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:41:32.342084 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.334439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:38297 (local address 127.6.4.2:37493)
0927 18:41:32.335122 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.335182 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.336488 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.336953 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.336959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.337042 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:32.337515 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.337527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.338251 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.338253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.339126 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.339135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.339654 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.339710 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.339882 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.340492 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:32.340494 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:32.340545 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:32.340621 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:32.340868 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:32.340876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:32.340882 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.340997 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.341265 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":132,"threads_started":1}
I20260927 18:41:32.343294 24040 heartbeater.cc:347] Connected to a master server at 127.6.4.62:38297
I20260927 18:41:32.343537 24040 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.344018 24040 heartbeater.cc:511] Master 127.6.4.62:38297 requested a full tablet report, sending...
I20260927 18:41:32.345018 23721 ts_manager.cc:194] Registered new tserver with Master: d89d3f01a5a14a6ab14c8ed7577b0423 (127.6.4.2:44933)
I20260927 18:41:32.345705 23721 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.2:37493
W20260927 18:41:32.452869 24043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:32.453127 24043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:32.453166 24043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:32.457139 24043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:32.457195 24043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:32.457249 24043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:32.461997 24043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:38297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:32.463018 24043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:32.464375 24043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:32.473409 24048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:32.473533 24051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.473519 24043 server_base.cc:1034] running on GCE node
W20260927 18:41:32.473412 24049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:32.474131 24043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:32.474709 24043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:32.475885 24043 hybrid_clock.cc:648] HybridClock initialized: now 1790534492475870 us; error 30 us; skew 500 ppm
I20260927 18:41:32.477643 24043 webserver.cc:460] Webserver started at http://127.6.4.3:44373/ using document root <none> and password file <none>
I20260927 18:41:32.478222 24043 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:32.478286 24043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:32.478472 24043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:32.480671 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "15efb747cf524f4e90074c08d666223e"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "cb410ce200449518cd624c89b2d6ef33"
server_key_iv: "53f3a0a04c07dde84a51a763277f3768"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.481218 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "15efb747cf524f4e90074c08d666223e"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "cb410ce200449518cd624c89b2d6ef33"
server_key_iv: "53f3a0a04c07dde84a51a763277f3768"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.484683 24043 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:32.486972 24057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.488180 24043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:32.488281 24043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "15efb747cf524f4e90074c08d666223e"
format_stamp: "Formatted at 2026-09-27 18:41:32 on dist-test-slave-bzzj"
server_key: "cb410ce200449518cd624c89b2d6ef33"
server_key_iv: "53f3a0a04c07dde84a51a763277f3768"
server_key_version: "encryptionkey@0"
I20260927 18:41:32.488373 24043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:32.508872 24043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:32.509542 24043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:32.509709 24043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:32.510301 24043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:32.511204 24043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:32.511250 24043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.511303 24043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:32.511328 24043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:32.519997 24043 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:33399
I20260927 18:41:32.520126 24170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:33399 every 8 connection(s)
I20260927 18:41:32.520951 24043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:32.528628 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24043
I20260927 18:41:32.528729 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:32.529009 6160 external_mini_cluster.cc:1428] Setting key e16b26c82a6ebf32e74866a398fcc519
I20260927 18:41:32.529665 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.522697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:54213 (local address 127.6.4.62:38297)
0927 18:41:32.522839 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.522844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.523883 (+ 1039us) server_negotiation.cc:407] Connection header received
0927 18:41:32.525229 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.525231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.525278 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.525354 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.526178 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.526696 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.527778 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.528000 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.528078 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.529184 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.529188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.529191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:32.529204 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.529226 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:32.529235 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.529316 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.529465 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.529526 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:41:32.530689 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.523119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:38297 (local address 127.6.4.3:54213)
0927 18:41:32.523726 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.523765 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.525045 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.525511 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.525517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.525600 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:32.526070 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.526082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.526801 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.526804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.527670 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.527679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.528087 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.528162 (+ 75us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.528309 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.528929 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:32.528931 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:32.528976 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:32.529038 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:32.529327 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:32.529335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:32.529341 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.529468 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.529734 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260927 18:41:32.531746 24171 heartbeater.cc:347] Connected to a master server at 127.6.4.62:38297
I20260927 18:41:32.531963 24171 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:32.532496 24171 heartbeater.cc:511] Master 127.6.4.62:38297 requested a full tablet report, sending...
I20260927 18:41:32.533392 23721 ts_manager.cc:194] Registered new tserver with Master: 15efb747cf524f4e90074c08d666223e (127.6.4.3:33399)
I20260927 18:41:32.533993 23721 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.3:54213
I20260927 18:41:32.542909 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:32.548199 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.544776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60848 (local address 127.6.4.62:38297)
0927 18:41:32.544902 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.544905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.545099 (+ 194us) server_negotiation.cc:407] Connection header received
0927 18:41:32.545188 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.545190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.545227 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.545290 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:32.545507 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.546024 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.546872 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.547087 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.547152 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.547824 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:32.547827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:32.547830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:32.547840 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:32.547858 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:32.547865 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:32.547917 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.548001 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.548049 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:41:32.550611 23721 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:32.553568 23721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:32.567447 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.562210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:44933 (local address 127.0.0.1:36598)
0927 18:41:32.563043 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.563054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.563137 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.563733 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.563735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.563749 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:32.563804 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.563808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.564765 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.564768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.565987 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.565997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.566880 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.566884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.566985 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.567047 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.567152 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.567273 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":59,"spinlock_wait_cycles":1792,"thread_start_us":84,"threads_started":1}
I20260927 18:41:32.567476 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.562605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.6.4.2:44933)
0927 18:41:32.562987 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.562999 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.563224 (+ 225us) server_negotiation.cc:407] Connection header received
0927 18:41:32.563298 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.563302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.563476 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.563614 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:32.563903 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.564651 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.566085 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.566758 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.566864 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.567056 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.567200 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.567301 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
I20260927 18:41:32.567911 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.561530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:46271 (local address 127.0.0.1:57266)
0927 18:41:32.562292 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.562345 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.562492 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.563099 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.563105 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.563135 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:32.563271 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.563278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.564732 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.564735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.566275 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.566286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.567453 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.567456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.567618 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.567645 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.567712 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.567781 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":148,"spinlock_wait_cycles":14080,"thread_start_us":78,"threads_started":1}
I20260927 18:41:32.567961 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.561679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.6.4.3:33399)
0927 18:41:32.562040 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.562052 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.562630 (+ 578us) server_negotiation.cc:407] Connection header received
0927 18:41:32.562713 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.562719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.562920 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.563465 (+ 545us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:32.563661 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.564758 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.566412 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.566975 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.567065 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.567235 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.567717 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.567807 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":136,"threads_started":1}
I20260927 18:41:32.567988 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.562167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.6.4.1:46271)
0927 18:41:32.562584 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.562595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.562618 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:41:32.562693 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.562698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.562910 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.563115 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:32.563409 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.564615 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.566398 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.567132 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.567227 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.567389 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.567758 (+ 369us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.567850 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":116,"threads_started":1}
I20260927 18:41:32.567639 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.561504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:33399 (local address 127.0.0.1:32848)
0927 18:41:32.562292 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.562343 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.562493 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.563438 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.563440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.563455 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:32.563560 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.563566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.564858 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.564861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.566275 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.566286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.567290 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.567294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.567375 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.567396 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.567459 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.567517 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":163,"spinlock_wait_cycles":2304,"thread_start_us":86,"threads_started":1}
I20260927 18:41:32.571112 24105 tablet_service.cc:1467] Processing CreateTablet for tablet af9a5e4b65724c87b42a6a4a94e877e5 (DEFAULT_TABLE table=test-table [id=ea5cd4f89289434c87a01d21149e5b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:32.570989 23843 tablet_service.cc:1467] Processing CreateTablet for tablet af9a5e4b65724c87b42a6a4a94e877e5 (DEFAULT_TABLE table=test-table [id=ea5cd4f89289434c87a01d21149e5b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:32.570987 23974 tablet_service.cc:1467] Processing CreateTablet for tablet af9a5e4b65724c87b42a6a4a94e877e5 (DEFAULT_TABLE table=test-table [id=ea5cd4f89289434c87a01d21149e5b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:32.571995 24105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9a5e4b65724c87b42a6a4a94e877e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:32.572013 23974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9a5e4b65724c87b42a6a4a94e877e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:32.572014 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9a5e4b65724c87b42a6a4a94e877e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:32.578034 24188 tablet_bootstrap.cc:492] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Bootstrap starting.
I20260927 18:41:32.579394 24190 tablet_bootstrap.cc:492] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: Bootstrap starting.
I20260927 18:41:32.579732 24189 tablet_bootstrap.cc:492] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Bootstrap starting.
I20260927 18:41:32.580274 24188 tablet_bootstrap.cc:654] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:32.580961 24188 log.cc:824] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:32.581986 24189 tablet_bootstrap.cc:654] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:32.581986 24190 tablet_bootstrap.cc:654] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:32.582687 24188 tablet_bootstrap.cc:492] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: No bootstrap required, opened a new log
I20260927 18:41:32.582816 24189 log.cc:824] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:32.582816 24190 log.cc:824] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:32.582916 24188 ts_tablet_manager.cc:1397] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:41:32.584830 24190 tablet_bootstrap.cc:492] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: No bootstrap required, opened a new log
I20260927 18:41:32.584830 24189 tablet_bootstrap.cc:492] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: No bootstrap required, opened a new log
I20260927 18:41:32.585024 24190 ts_tablet_manager.cc:1397] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:32.585024 24189 ts_tablet_manager.cc:1397] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:41:32.586490 24188 raft_consensus.cc:348] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.586726 24188 raft_consensus.cc:374] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:32.586795 24188 raft_consensus.cc:729] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15efb747cf524f4e90074c08d666223e, State: Initialized, Role: FOLLOWER
I20260927 18:41:32.587272 24188 consensus_queue.cc:260] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.587880 24171 heartbeater.cc:503] Master 127.6.4.62:38297 was elected leader, sending a full tablet report...
I20260927 18:41:32.588269 24188 ts_tablet_manager.cc:1428] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:41:32.589216 24189 raft_consensus.cc:348] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.589216 24190 raft_consensus.cc:348] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.589478 24189 raft_consensus.cc:374] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:32.589572 24189 raft_consensus.cc:729] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d89d3f01a5a14a6ab14c8ed7577b0423, State: Initialized, Role: FOLLOWER
I20260927 18:41:32.589783 24190 raft_consensus.cc:374] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:32.589880 24190 raft_consensus.cc:729] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77777c4dc30c435aaca0ee59ce98ad02, State: Initialized, Role: FOLLOWER
I20260927 18:41:32.590050 24189 consensus_queue.cc:260] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.590277 24190 consensus_queue.cc:260] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.590927 24040 heartbeater.cc:503] Master 127.6.4.62:38297 was elected leader, sending a full tablet report...
I20260927 18:41:32.591176 24190 ts_tablet_manager.cc:1428] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:32.591187 24189 ts_tablet_manager.cc:1428] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 18:41:32.591495 23909 heartbeater.cc:503] Master 127.6.4.62:38297 was elected leader, sending a full tablet report...
W20260927 18:41:32.644392 23910 tablet.cc:2369] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:32.672777 24196 raft_consensus.cc:482] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:32.672986 24196 raft_consensus.cc:504] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.674154 24196 leader_election.cc:288] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15efb747cf524f4e90074c08d666223e (127.6.4.3:33399), d89d3f01a5a14a6ab14c8ed7577b0423 (127.6.4.2:44933)
I20260927 18:41:32.677707 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.674456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:44671 (local address 127.6.4.3:33399)
0927 18:41:32.674596 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.674600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.674726 (+ 126us) server_negotiation.cc:407] Connection header received
0927 18:41:32.674863 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.674866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.674905 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.674973 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:32.675254 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.675771 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.676844 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.677217 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.677302 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.677358 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.677523 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.677587 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 18:41:32.677733 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.674517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:41017 (local address 127.6.4.2:44933)
0927 18:41:32.674671 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.674676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.674757 (+ 81us) server_negotiation.cc:407] Connection header received
0927 18:41:32.674888 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.674890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.674931 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.674989 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:32.675266 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.675743 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.676868 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.677236 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.677322 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.677376 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.677552 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.677613 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 18:41:32.677765 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.674407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:44933 (local address 127.6.4.1:41017)
0927 18:41:32.674670 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.674683 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.674789 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.675012 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.675015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.675038 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:32.675161 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.675166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.675855 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.675858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.676744 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.676752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.677355 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.677359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.677444 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.677461 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.677522 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.677585 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":27,"spinlock_wait_cycles":768,"thread_start_us":77,"threads_started":1}
I20260927 18:41:32.677764 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.674344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:33399 (local address 127.6.4.1:44671)
0927 18:41:32.674639 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.674653 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:32.674760 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:32.674992 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:32.674994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:32.675009 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:32.675153 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.675158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.675866 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.675870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.676728 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:32.676736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.677319 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.677322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:32.677411 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.677432 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:41:32.677494 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:41:32.677562 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":5,"spinlock_wait_cycles":24064,"thread_start_us":71,"threads_started":1}
I20260927 18:41:32.678416 24125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9a5e4b65724c87b42a6a4a94e877e5" candidate_uuid: "77777c4dc30c435aaca0ee59ce98ad02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15efb747cf524f4e90074c08d666223e" is_pre_election: true
I20260927 18:41:32.678416 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9a5e4b65724c87b42a6a4a94e877e5" candidate_uuid: "77777c4dc30c435aaca0ee59ce98ad02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" is_pre_election: true
I20260927 18:41:32.678673 23994 raft_consensus.cc:2454] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77777c4dc30c435aaca0ee59ce98ad02 in term 0.
I20260927 18:41:32.678678 24125 raft_consensus.cc:2454] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77777c4dc30c435aaca0ee59ce98ad02 in term 0.
I20260927 18:41:32.679097 23797 leader_election.cc:302] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 15efb747cf524f4e90074c08d666223e, 77777c4dc30c435aaca0ee59ce98ad02, d89d3f01a5a14a6ab14c8ed7577b0423; no voters:
I20260927 18:41:32.679333 24196 raft_consensus.cc:2781] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:32.679404 24196 raft_consensus.cc:482] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:32.679435 24196 raft_consensus.cc:3037] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:32.680478 24196 raft_consensus.cc:504] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.680825 24196 leader_election.cc:288] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [CANDIDATE]: Term 1 election: Requested vote from peers 15efb747cf524f4e90074c08d666223e (127.6.4.3:33399), d89d3f01a5a14a6ab14c8ed7577b0423 (127.6.4.2:44933)
I20260927 18:41:32.681143 23994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9a5e4b65724c87b42a6a4a94e877e5" candidate_uuid: "77777c4dc30c435aaca0ee59ce98ad02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423"
I20260927 18:41:32.681133 24125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9a5e4b65724c87b42a6a4a94e877e5" candidate_uuid: "77777c4dc30c435aaca0ee59ce98ad02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15efb747cf524f4e90074c08d666223e"
I20260927 18:41:32.681248 23994 raft_consensus.cc:3037] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:32.681258 24125 raft_consensus.cc:3037] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:32.682407 24125 raft_consensus.cc:2454] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77777c4dc30c435aaca0ee59ce98ad02 in term 1.
I20260927 18:41:32.682407 23994 raft_consensus.cc:2454] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77777c4dc30c435aaca0ee59ce98ad02 in term 1.
I20260927 18:41:32.682689 23797 leader_election.cc:302] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77777c4dc30c435aaca0ee59ce98ad02, d89d3f01a5a14a6ab14c8ed7577b0423; no voters:
I20260927 18:41:32.682842 24196 raft_consensus.cc:2781] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:32.683039 24196 raft_consensus.cc:686] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [term 1 LEADER]: Becoming Leader. State: Replica: 77777c4dc30c435aaca0ee59ce98ad02, State: Running, Role: LEADER
I20260927 18:41:32.683372 24196 consensus_queue.cc:237] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } }
I20260927 18:41:32.686295 23720 catalog_manager.cc:5495] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77777c4dc30c435aaca0ee59ce98ad02 (127.6.4.1). New cstate: current_term: 1 leader_uuid: "77777c4dc30c435aaca0ee59ce98ad02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77777c4dc30c435aaca0ee59ce98ad02" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 46271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:32.718762 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:32.715552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57272 (local address 127.6.4.1:46271)
0927 18:41:32.715659 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:32.715663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:32.715676 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:41:32.715778 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:32.715781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:32.715842 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:32.715927 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:32.716131 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.716628 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.717494 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:32.717702 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:32.717785 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:32.718058 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:32.718478 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:32.718564 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:32.718610 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:41:32.718654 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:41:32.724380 24125 raft_consensus.cc:1261] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 1 FOLLOWER]: Refusing update from remote peer 77777c4dc30c435aaca0ee59ce98ad02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:32.724388 23994 raft_consensus.cc:1261] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 1 FOLLOWER]: Refusing update from remote peer 77777c4dc30c435aaca0ee59ce98ad02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:32.725010 24199 consensus_queue.cc:1035] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d89d3f01a5a14a6ab14c8ed7577b0423" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:32.725191 24196 consensus_queue.cc:1035] T af9a5e4b65724c87b42a6a4a94e877e5 P 77777c4dc30c435aaca0ee59ce98ad02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15efb747cf524f4e90074c08d666223e" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:32.731901 24202 mvcc.cc:205] Tried to move back new op lower bound from 7334029282192420864 to 7334029282033725440. Current Snapshot: MvccSnapshot[applied={T|T < 7334029282192420864}]
I20260927 18:41:32.732316 24204 mvcc.cc:205] Tried to move back new op lower bound from 7334029282192420864 to 7334029282033725440. Current Snapshot: MvccSnapshot[applied={T|T < 7334029282192420864}]
I20260927 18:41:32.740468 23721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:32.740617 23721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:32.742847 23721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 207c46d596fb49968fd4d61482e2911f: Sending DeleteTablet for 3 replicas of tablet af9a5e4b65724c87b42a6a4a94e877e5
I20260927 18:41:32.743592 24105 tablet_service.cc:1514] Processing DeleteTablet for tablet af9a5e4b65724c87b42a6a4a94e877e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:32 UTC) from {username='slave'} at 127.0.0.1:32848
I20260927 18:41:32.743649 23974 tablet_service.cc:1514] Processing DeleteTablet for tablet af9a5e4b65724c87b42a6a4a94e877e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:32 UTC) from {username='slave'} at 127.0.0.1:36598
I20260927 18:41:32.743675 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet af9a5e4b65724c87b42a6a4a94e877e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:32 UTC) from {username='slave'} at 127.0.0.1:57266
I20260927 18:41:32.744158 24212 tablet_replica.cc:330] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: stopping tablet replica
I20260927 18:41:32.744266 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23781
I20260927 18:41:32.744952 24212 raft_consensus.cc:2229] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:32.745262 24212 raft_consensus.cc:2258] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:32.746694 24212 ts_tablet_manager.cc:1905] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:32.745712 24211 tablet_replica.cc:330] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: stopping tablet replica
I20260927 18:41:32.747099 24211 raft_consensus.cc:2229] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:32.747365 24211 raft_consensus.cc:2258] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:32.748502 24211 ts_tablet_manager.cc:1905] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:32.749354 24212 ts_tablet_manager.cc:1918] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:32.749456 24212 log.cc:1196] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/af9a5e4b65724c87b42a6a4a94e877e5
I20260927 18:41:32.749785 24212 ts_tablet_manager.cc:1939] T af9a5e4b65724c87b42a6a4a94e877e5 P 15efb747cf524f4e90074c08d666223e: Deleting consensus metadata
I20260927 18:41:32.750522 24211 ts_tablet_manager.cc:1918] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:32.750599 24211 log.cc:1196] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/af9a5e4b65724c87b42a6a4a94e877e5
I20260927 18:41:32.750878 24211 ts_tablet_manager.cc:1939] T af9a5e4b65724c87b42a6a4a94e877e5 P d89d3f01a5a14a6ab14c8ed7577b0423: Deleting consensus metadata
I20260927 18:41:32.750994 23705 catalog_manager.cc:4841] TS 15efb747cf524f4e90074c08d666223e (127.6.4.3:33399): tablet af9a5e4b65724c87b42a6a4a94e877e5 (table test-table [id=ea5cd4f89289434c87a01d21149e5b6e]) successfully deleted
W20260927 18:41:32.751593 23706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:46271 (error 108)
I20260927 18:41:32.752066 23706 catalog_manager.cc:4841] TS d89d3f01a5a14a6ab14c8ed7577b0423 (127.6.4.2:44933): tablet af9a5e4b65724c87b42a6a4a94e877e5 (table test-table [id=ea5cd4f89289434c87a01d21149e5b6e]) successfully deleted
W20260927 18:41:32.752607 24186 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:41:32.752367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:46271 (local address 127.0.0.1:57282)
0927 18:41:32.752474 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:32.752534 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:46271: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:41:32.752759 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23912
W20260927 18:41:32.752733 23706 catalog_manager.cc:4584] TS 77777c4dc30c435aaca0ee59ce98ad02 (127.6.4.1:46271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af9a5e4b65724c87b42a6a4a94e877e5: Network error: Client connection negotiation failed: client connection to 127.6.4.1:46271: connect: Connection reset by peer (error 104)
I20260927 18:41:32.758275 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24043
I20260927 18:41:32.762421 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 23690
2026-09-27T18:41:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1042 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:32 dist-test-slave-bzzj krb5kdc[24217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:32 dist-test-slave-bzzj krb5kdc[24217](info): set up 2 sockets
Sep 27 18:41:32 dist-test-slave-bzzj krb5kdc[24217](info): commencing operation
krb5kdc: starting...
W20260927 18:41:33.703634 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:33 dist-test-slave-bzzj krb5kdc[24217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:41:33.817063 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:36467
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:36467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:33.924882 24230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:33.925159 24230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:33.925216 24230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:33.929302 24230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:33.929368 24230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:33.929394 24230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:33.929416 24230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:33.929445 24230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:33.934815 24230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:36467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:36467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:33.935896 24230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:33.936952 24230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:41:33.945058 24230 server_base.cc:1034] running on GCE node
W20260927 18:41:33.945042 24236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:33.945194 24238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:33.945019 24235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:33.945756 24230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:33.946835 24230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:33.947984 24230 hybrid_clock.cc:648] HybridClock initialized: now 1790534493947961 us; error 41 us; skew 500 ppm
Sep 27 18:41:33 dist-test-slave-bzzj krb5kdc[24217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534493, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:33.950591 24230 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:33.951452 24230 webserver.cc:460] Webserver started at http://127.6.4.62:45573/ using document root <none> and password file <none>
I20260927 18:41:33.951985 24230 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:33.952049 24230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:33.952270 24230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:33.954310 24230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "1b49935e832040459b73f5a96a4bbb46"
format_stamp: "Formatted at 2026-09-27 18:41:33 on dist-test-slave-bzzj"
server_key: "1ec9f7de038f0ce0024a904581bbbc3a"
server_key_iv: "105f9e270da9af479dc151a80457588e"
server_key_version: "encryptionkey@0"
I20260927 18:41:33.954965 24230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "1b49935e832040459b73f5a96a4bbb46"
format_stamp: "Formatted at 2026-09-27 18:41:33 on dist-test-slave-bzzj"
server_key: "1ec9f7de038f0ce0024a904581bbbc3a"
server_key_iv: "105f9e270da9af479dc151a80457588e"
server_key_version: "encryptionkey@0"
I20260927 18:41:33.959257 24230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260927 18:41:33.961565 24245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:33.962785 24230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:33.962895 24230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "1b49935e832040459b73f5a96a4bbb46"
format_stamp: "Formatted at 2026-09-27 18:41:33 on dist-test-slave-bzzj"
server_key: "1ec9f7de038f0ce0024a904581bbbc3a"
server_key_iv: "105f9e270da9af479dc151a80457588e"
server_key_version: "encryptionkey@0"
I20260927 18:41:33.962989 24230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:33.982184 24230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:33.985518 24230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:33.985723 24230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:33.993364 24230 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:36467
I20260927 18:41:33.993381 24297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:36467 every 8 connection(s)
I20260927 18:41:33.994495 24230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:33.997931 24298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:34.002530 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24230
I20260927 18:41:34.002658 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:34.002976 6160 external_mini_cluster.cc:1428] Setting key 34e3ddf429a526ca2860ba6fab919610
I20260927 18:41:34.004969 24298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46: Bootstrap starting.
I20260927 18:41:34.007601 24298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:34.008687 24298 log.cc:824] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:34.011305 24298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46: No bootstrap required, opened a new log
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:34.016238 24298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b49935e832040459b73f5a96a4bbb46" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36467 } }
I20260927 18:41:34.016571 24298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:34.016634 24298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b49935e832040459b73f5a96a4bbb46, State: Initialized, Role: FOLLOWER
I20260927 18:41:34.017114 24298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b49935e832040459b73f5a96a4bbb46" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36467 } }
I20260927 18:41:34.017258 24298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:34.017354 24298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:34.017465 24298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:34.018657 24298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b49935e832040459b73f5a96a4bbb46" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36467 } }
I20260927 18:41:34.019039 24298 leader_election.cc:302] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b49935e832040459b73f5a96a4bbb46; no voters:
I20260927 18:41:34.019392 24298 leader_election.cc:288] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:34.019513 24303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:34.019704 24303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [term 1 LEADER]: Becoming Leader. State: Replica: 1b49935e832040459b73f5a96a4bbb46, State: Running, Role: LEADER
I20260927 18:41:34.019795 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.004712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.6.4.62:36467)
0927 18:41:34.005210 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.005227 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.005272 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:41:34.006010 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.006031 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.006453 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.006946 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:34.007903 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.009444 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.010474 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.011193 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.011382 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.014327 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:34.014336 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:34.014360 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:34.014400 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:34.016504 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.016971 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.016976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.016981 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.017101 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.017399 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.017403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.017405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.017946 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:34.018136 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.018415 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.018741 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":149,"threads_started":1}
I20260927 18:41:34.020052 24303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b49935e832040459b73f5a96a4bbb46" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36467 } }
I20260927 18:41:34.020695 24298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:34.021845 24304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b49935e832040459b73f5a96a4bbb46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b49935e832040459b73f5a96a4bbb46" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36467 } } }
I20260927 18:41:34.021898 24305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b49935e832040459b73f5a96a4bbb46. Latest consensus state: current_term: 1 leader_uuid: "1b49935e832040459b73f5a96a4bbb46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b49935e832040459b73f5a96a4bbb46" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36467 } } }
I20260927 18:41:34.021970 24304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:34.022003 24305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:34.022393 24307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:34.025578 24307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:34.030696 24307 catalog_manager.cc:1347] Generated new cluster ID: 5bb7cac1acd34533a17459bcdc4d55c7
I20260927 18:41:34.030767 24307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:34.043943 24307 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:34.044951 24307 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:34.055722 24307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46: Generated new TSK 0
I20260927 18:41:34.056442 24307 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:41:34.080566 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36467
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:34.188463 24324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:34.188737 24324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:34.188797 24324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:34.192847 24324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:34.192911 24324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:34.192992 24324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:34.198305 24324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:34.199339 24324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:34.200381 24324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:34.209425 24329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:34.209430 24330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:34.209501 24324 server_base.cc:1034] running on GCE node
W20260927 18:41:34.209695 24332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:34.210074 24324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:34.210686 24324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:34.211854 24324 hybrid_clock.cc:648] HybridClock initialized: now 1790534494211836 us; error 31 us; skew 500 ppm
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:34.214498 24324 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:41:34.215343 24324 webserver.cc:460] Webserver started at http://127.6.4.1:39071/ using document root <none> and password file <none>
I20260927 18:41:34.215866 24324 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:34.215936 24324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:34.216176 24324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:34.218209 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "a4e03abe93f240dfb347069c17ef7b09"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "cf96c02a4c7e9f50cc44a17e79fdcea9"
server_key_iv: "927fd8d5d2ec1055c99735c9b893dce9"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.218715 24324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "a4e03abe93f240dfb347069c17ef7b09"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "cf96c02a4c7e9f50cc44a17e79fdcea9"
server_key_iv: "927fd8d5d2ec1055c99735c9b893dce9"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.222254 24324 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:34.224743 24339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.225960 24324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:34.226070 24324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "a4e03abe93f240dfb347069c17ef7b09"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "cf96c02a4c7e9f50cc44a17e79fdcea9"
server_key_iv: "927fd8d5d2ec1055c99735c9b893dce9"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.226171 24324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:34.251574 24324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:34.254431 24324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:34.254608 24324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:34.255196 24324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:34.256052 24324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:34.256129 24324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.256194 24324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:34.256222 24324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.264422 24324 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:36225
I20260927 18:41:34.264452 24452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:36225 every 8 connection(s)
I20260927 18:41:34.265275 24324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:34.266942 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24324
I20260927 18:41:34.267067 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:34.267344 6160 external_mini_cluster.cc:1428] Setting key e5bcea006654b57ae66e8b5453d7e483
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:41:34.278055 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.266990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:41139 (local address 127.6.4.62:36467)
0927 18:41:34.267113 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.267117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.268069 (+ 952us) server_negotiation.cc:407] Connection header received
0927 18:41:34.269107 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.269110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.269158 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.269225 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:34.270337 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.270858 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.271963 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.272240 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.272334 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.274789 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:34.274792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:34.274795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:34.274820 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:34.276120 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.276588 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.276592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.276594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.276664 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.276963 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.276966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.276968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.277148 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:34.277230 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.277767 (+ 537us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.277887 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:41:34.278860 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.267413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36467 (local address 127.6.4.1:41139)
0927 18:41:34.267913 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:34.267972 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:34.268918 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:34.269387 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:34.269393 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:34.269724 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:34.270199 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.270212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.270970 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.270973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.271852 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.271861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.272320 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.272374 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.272532 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.273087 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:34.273102 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:34.274632 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:34.276215 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:34.276220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:34.276236 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:34.276503 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:34.276740 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:34.276743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:34.276745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:34.276883 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:34.277274 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:34.277281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:34.277516 (+ 235us) client_negotiation.cc:769] Sending connection context
0927 18:41:34.277714 (+ 198us) client_negotiation.cc:240] Negotiation successful
0927 18:41:34.278039 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":110,"threads_started":1}
I20260927 18:41:34.280062 24453 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36467
I20260927 18:41:34.280349 24453 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:34.280856 24453 heartbeater.cc:511] Master 127.6.4.62:36467 requested a full tablet report, sending...
I20260927 18:41:34.282383 24262 ts_manager.cc:194] Registered new tserver with Master: a4e03abe93f240dfb347069c17ef7b09 (127.6.4.1:36225)
I20260927 18:41:34.283473 24262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:41139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:41:34.294507 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36467
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:34.403712 24458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:34.403990 24458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:34.404047 24458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:34.408000 24458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:34.408061 24458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:34.408162 24458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:34.413298 24458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:34.414336 24458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:34.415305 24458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:34.424547 24464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:34.424531 24463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:34.424623 24458 server_base.cc:1034] running on GCE node
W20260927 18:41:34.424520 24466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:34.425148 24458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:34.425719 24458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:34.426870 24458 hybrid_clock.cc:648] HybridClock initialized: now 1790534494426848 us; error 38 us; skew 500 ppm
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:34.429410 24458 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:41:34.430260 24458 webserver.cc:460] Webserver started at http://127.6.4.2:34905/ using document root <none> and password file <none>
I20260927 18:41:34.430809 24458 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:34.430877 24458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:34.431063 24458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:34.433041 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "ca91975311ac2b71c28734969b56cbe6"
server_key_iv: "97c9c2f0337f26d7870ccc49d96207d9"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.433548 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "ca91975311ac2b71c28734969b56cbe6"
server_key_iv: "97c9c2f0337f26d7870ccc49d96207d9"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.437131 24458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:34.439678 24473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.440886 24458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260927 18:41:34.440989 24458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "ca91975311ac2b71c28734969b56cbe6"
server_key_iv: "97c9c2f0337f26d7870ccc49d96207d9"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.441083 24458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:34.451099 24458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:34.453868 24458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:34.454037 24458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:34.454595 24458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:34.455385 24458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:34.455430 24458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.455483 24458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:34.455509 24458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.464051 24458 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:41273
I20260927 18:41:34.464071 24586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:41273 every 8 connection(s)
I20260927 18:41:34.464956 24458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:34.470152 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24458
I20260927 18:41:34.470264 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:34.470531 6160 external_mini_cluster.cc:1428] Setting key e0bbbd793b86015be8ad1ebcb17ce1cc
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:34.478132 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.466703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:52505 (local address 127.6.4.62:36467)
0927 18:41:34.466861 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.466865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.467736 (+ 871us) server_negotiation.cc:407] Connection header received
0927 18:41:34.468620 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.468622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.468669 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.468739 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:34.469850 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.470431 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.471689 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.471914 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.471999 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.474336 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:34.474340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:34.474342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:34.474366 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:34.476014 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.476622 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.476625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.476626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.476698 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.477055 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.477057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.477058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.477227 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:34.477296 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.477894 (+ 598us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.478008 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 18:41:34.479295 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.467128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36467 (local address 127.6.4.2:52505)
0927 18:41:34.467596 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:34.467637 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:34.468437 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:34.468909 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:34.468915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:34.469253 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:34.469716 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.469729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.470546 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.470550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.471574 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.471583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.472002 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.472056 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.472246 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.472791 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:34.472807 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:34.474155 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:34.476155 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:34.476162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:34.476180 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:34.476531 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:34.476781 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:34.476785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:34.476786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:34.476969 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:34.477319 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:34.477326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:34.477654 (+ 328us) client_negotiation.cc:769] Sending connection context
0927 18:41:34.477863 (+ 209us) client_negotiation.cc:240] Negotiation successful
0927 18:41:34.478209 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
I20260927 18:41:34.480566 24587 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36467
I20260927 18:41:34.480836 24587 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:34.481312 24587 heartbeater.cc:511] Master 127.6.4.62:36467 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:41:34.482388 24262 ts_manager.cc:194] Registered new tserver with Master: 55bb4a3b7d3b458ebcfd9d9c77085bf9 (127.6.4.2:41273)
I20260927 18:41:34.483107 24262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:52505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:41:34.496367 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36467
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:34.604797 24592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:34.605046 24592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:34.605087 24592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:34.609033 24592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:34.609090 24592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:34.609153 24592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:34.614274 24592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:34.615262 24592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:34.616243 24592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:34.625509 24597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:34.625684 24592 server_base.cc:1034] running on GCE node
W20260927 18:41:34.625509 24600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:34.625509 24598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:34.626116 24592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:34.626705 24592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:34.627871 24592 hybrid_clock.cc:648] HybridClock initialized: now 1790534494627844 us; error 37 us; skew 500 ppm
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:34.630403 24592 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:41:34.631250 24592 webserver.cc:460] Webserver started at http://127.6.4.3:38485/ using document root <none> and password file <none>
I20260927 18:41:34.631772 24592 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:34.631840 24592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:34.632019 24592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:34.634008 24592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "6d0d953551dd4556914f1c0cb12665b6"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "d905d00263b44f57b6853f0c418e51d6"
server_key_iv: "a40dcf971deebe0187b98fe1d87a0f69"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.634506 24592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "6d0d953551dd4556914f1c0cb12665b6"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "d905d00263b44f57b6853f0c418e51d6"
server_key_iv: "a40dcf971deebe0187b98fe1d87a0f69"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.638037 24592 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:34.640336 24607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.641517 24592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:34.641618 24592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "6d0d953551dd4556914f1c0cb12665b6"
format_stamp: "Formatted at 2026-09-27 18:41:34 on dist-test-slave-bzzj"
server_key: "d905d00263b44f57b6853f0c418e51d6"
server_key_iv: "a40dcf971deebe0187b98fe1d87a0f69"
server_key_version: "encryptionkey@0"
I20260927 18:41:34.641709 24592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:34.659283 24592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:34.662019 24592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:34.662194 24592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:34.662750 24592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:34.663524 24592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:34.663568 24592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.663619 24592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:34.663647 24592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:34.671641 24592 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:41231
I20260927 18:41:34.671690 24720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:41231 every 8 connection(s)
I20260927 18:41:34.672503 24592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:41:34 dist-test-slave-bzzj krb5kdc[24217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534494, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:34.682034 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24592
I20260927 18:41:34.682134 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:34.682452 6160 external_mini_cluster.cc:1428] Setting key f32ffa28499e657d9caf15266ba47bfc
I20260927 18:41:34.684818 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.674167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:54513 (local address 127.6.4.62:36467)
0927 18:41:34.674297 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.674301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.675093 (+ 792us) server_negotiation.cc:407] Connection header received
0927 18:41:34.675949 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.675953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.676006 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.676083 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:34.677225 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.677747 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.678850 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.679063 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.679150 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.681407 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:34.681410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:34.681412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:34.681434 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:34.682812 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.683264 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.683267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.683269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.683365 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.683764 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.683767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.683769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.683949 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:34.684021 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.684602 (+ 581us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.684700 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:41:34.685590 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.674525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36467 (local address 127.6.4.3:54513)
0927 18:41:34.674953 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:34.674994 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:34.675753 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:34.676280 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:34.676287 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:34.676625 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:34.677091 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.677104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.677865 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.677867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.678734 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.678743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.679144 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.679197 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.679351 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.679880 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:34.679895 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:34.681251 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:34.682914 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:34.682920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:34.682935 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:34.683176 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:34.683479 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:34.683482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:34.683484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:34.683679 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:34.684045 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:34.684052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:34.684388 (+ 336us) client_negotiation.cc:769] Sending connection context
0927 18:41:34.684551 (+ 163us) client_negotiation.cc:240] Negotiation successful
0927 18:41:34.684804 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
I20260927 18:41:34.686578 24721 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36467
I20260927 18:41:34.686815 24721 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:34.687275 24721 heartbeater.cc:511] Master 127.6.4.62:36467 requested a full tablet report, sending...
I20260927 18:41:34.688256 24262 ts_manager.cc:194] Registered new tserver with Master: 6d0d953551dd4556914f1c0cb12665b6 (127.6.4.3:41231)
I20260927 18:41:34.688797 24262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:54513
I20260927 18:41:34.696084 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:34.704465 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.697929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.6.4.62:36467)
0927 18:41:34.698035 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.698039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.698140 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:41:34.698231 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.698233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.698269 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.698341 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:34.698927 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.699422 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.700223 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.700435 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.700516 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.701960 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:34.701963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:34.701965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:34.701987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:34.703038 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.703444 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.703446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.703447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.703512 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:34.703728 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:34.703730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:34.703732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:34.703934 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:34.704044 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.704274 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.704346 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:41:34.707108 24262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:34.710057 24262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:34.723831 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.718020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41273 (local address 127.0.0.1:48630)
0927 18:41:34.718743 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:34.718756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:34.718841 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:34.719580 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:34.719586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:34.719613 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:34.719897 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.719906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.720944 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.720948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.722314 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.722326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.723286 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.723289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.723406 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.723442 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 18:41:34.723546 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 18:41:34.723673 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":101,"spinlock_wait_cycles":768,"thread_start_us":73,"threads_started":1}
I20260927 18:41:34.723837 24736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.718528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.6.4.2:41273)
0927 18:41:34.718903 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.718915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.718937 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:41:34.719024 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.719028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.719240 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.719392 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:34.720017 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.720826 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.722453 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.723175 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.723285 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.723441 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.723585 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.723676 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":98,"threads_started":1}
I20260927 18:41:34.724115 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.717716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36225 (local address 127.0.0.1:51106)
0927 18:41:34.718046 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:34.718091 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:34.718206 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:34.719589 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:34.719592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:34.719629 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:34.719926 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.719932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.721398 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.721402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.722590 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.722599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.723713 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.723716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.723801 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.723822 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:41:34.723869 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:41:34.723936 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":199,"spinlock_wait_cycles":11008,"thread_start_us":79,"threads_started":1}
I20260927 18:41:34.724501 24734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.717811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:41231 (local address 127.0.0.1:48942)
0927 18:41:34.718532 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:34.718545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:34.718833 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:34.719717 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:34.719719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:34.719733 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:34.719962 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.719970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.721569 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.721572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.722700 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:34.722709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.723869 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.723872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:34.723956 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.723974 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:41:34.724034 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:41:34.724366 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":116,"thread_start_us":89,"threads_started":1}
I20260927 18:41:34.724715 24735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.718385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.6.4.1:36225)
0927 18:41:34.719214 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.719223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.719241 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:41:34.719299 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.719314 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.719454 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.719754 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:34.720043 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.721286 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.722700 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.723580 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.724293 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.724431 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.724478 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.724571 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":102,"threads_started":1}
I20260927 18:41:34.724881 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:34.718707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.6.4.3:41231)
0927 18:41:34.719142 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:34.719153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:34.719175 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:41:34.719249 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:34.719254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:34.719427 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:34.719569 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:34.720064 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.721274 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.722810 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:34.723580 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:34.723681 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:34.723815 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:34.724177 (+ 362us) server_negotiation.cc:299] Negotiation successful
0927 18:41:34.724750 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":138,"threads_started":1}
I20260927 18:41:34.727418 24387 tablet_service.cc:1467] Processing CreateTablet for tablet 1139d41659f9454bb02f30621f2643d0 (DEFAULT_TABLE table=test-table [id=0d715f34784c41cb8ac632aba087218e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:34.727451 24521 tablet_service.cc:1467] Processing CreateTablet for tablet 1139d41659f9454bb02f30621f2643d0 (DEFAULT_TABLE table=test-table [id=0d715f34784c41cb8ac632aba087218e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:34.727665 24655 tablet_service.cc:1467] Processing CreateTablet for tablet 1139d41659f9454bb02f30621f2643d0 (DEFAULT_TABLE table=test-table [id=0d715f34784c41cb8ac632aba087218e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:34.728412 24387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1139d41659f9454bb02f30621f2643d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:34.728513 24521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1139d41659f9454bb02f30621f2643d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:34.728509 24655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1139d41659f9454bb02f30621f2643d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:34.734540 24738 tablet_bootstrap.cc:492] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: Bootstrap starting.
I20260927 18:41:34.736063 24740 tablet_bootstrap.cc:492] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Bootstrap starting.
I20260927 18:41:34.736393 24739 tablet_bootstrap.cc:492] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Bootstrap starting.
I20260927 18:41:34.737170 24738 tablet_bootstrap.cc:654] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:34.738113 24740 tablet_bootstrap.cc:654] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:34.738387 24738 log.cc:824] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:34.738794 24739 tablet_bootstrap.cc:654] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:34.739176 24740 log.cc:824] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:34.739532 24739 log.cc:824] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:34.740525 24738 tablet_bootstrap.cc:492] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: No bootstrap required, opened a new log
I20260927 18:41:34.740718 24738 ts_tablet_manager.cc:1397] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:34.741142 24740 tablet_bootstrap.cc:492] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: No bootstrap required, opened a new log
I20260927 18:41:34.741185 24739 tablet_bootstrap.cc:492] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: No bootstrap required, opened a new log
I20260927 18:41:34.741317 24740 ts_tablet_manager.cc:1397] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:41:34.741348 24739 ts_tablet_manager.cc:1397] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:41:34.744553 24739 raft_consensus.cc:348] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:34.744788 24739 raft_consensus.cc:374] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:34.744875 24739 raft_consensus.cc:729] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55bb4a3b7d3b458ebcfd9d9c77085bf9, State: Initialized, Role: FOLLOWER
I20260927 18:41:34.744650 24738 raft_consensus.cc:348] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:34.745020 24738 raft_consensus.cc:374] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:34.744905 24740 raft_consensus.cc:348] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:34.745122 24738 raft_consensus.cc:729] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e03abe93f240dfb347069c17ef7b09, State: Initialized, Role: FOLLOWER
I20260927 18:41:34.745188 24740 raft_consensus.cc:374] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:34.745239 24740 raft_consensus.cc:729] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0d953551dd4556914f1c0cb12665b6, State: Initialized, Role: FOLLOWER
I20260927 18:41:34.745332 24739 consensus_queue.cc:260] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:34.745548 24738 consensus_queue.cc:260] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:34.745605 24740 consensus_queue.cc:260] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:34.746232 24453 heartbeater.cc:503] Master 127.6.4.62:36467 was elected leader, sending a full tablet report...
I20260927 18:41:34.746417 24739 ts_tablet_manager.cc:1428] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:41:34.746505 24738 ts_tablet_manager.cc:1428] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 18:41:34.746717 24587 heartbeater.cc:503] Master 127.6.4.62:36467 was elected leader, sending a full tablet report...
I20260927 18:41:34.746950 24721 heartbeater.cc:503] Master 127.6.4.62:36467 was elected leader, sending a full tablet report...
I20260927 18:41:34.748147 24740 ts_tablet_manager.cc:1428] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260927 18:41:34.766430 24454 tablet.cc:2369] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:41:34.923372 24722 tablet.cc:2369] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:41:34.966055 24588 tablet.cc:2369] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:35.083510 24746 raft_consensus.cc:482] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:35.083709 24746 raft_consensus.cc:504] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:35.084870 24746 leader_election.cc:288] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4e03abe93f240dfb347069c17ef7b09 (127.6.4.1:36225), 55bb4a3b7d3b458ebcfd9d9c77085bf9 (127.6.4.2:41273)
I20260927 18:41:35.088490 24736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.085217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:50135 (local address 127.6.4.2:41273)
0927 18:41:35.085325 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:35.085329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:35.085546 (+ 217us) server_negotiation.cc:407] Connection header received
0927 18:41:35.085633 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:35.085636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:35.085678 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:35.085741 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:35.086003 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.086507 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.087587 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.087966 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.088054 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.088138 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:35.088306 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 18:41:35.088370 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:41:35.088490 24735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.085167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:45131 (local address 127.6.4.1:36225)
0927 18:41:35.085287 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:35.085292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:35.085378 (+ 86us) server_negotiation.cc:407] Connection header received
0927 18:41:35.085497 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:35.085499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:35.085545 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:35.085617 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:35.086045 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.086528 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.087586 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.087963 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.088051 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.088142 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:35.088297 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 18:41:35.088372 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:41:35.088501 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.085083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36225 (local address 127.6.4.3:45131)
0927 18:41:35.085265 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:35.085280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:35.085395 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:35.085824 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:35.085826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:35.085848 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:35.085945 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.085949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.086625 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.086629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.087479 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.087487 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.088066 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.088070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.088191 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.088210 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 18:41:35.088272 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:41:35.088350 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":30}
I20260927 18:41:35.088544 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.085118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41273 (local address 127.6.4.3:50135)
0927 18:41:35.085461 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:35.085472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:35.085558 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:35.085780 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:35.085782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:35.085794 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:35.085892 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.085896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.086610 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.086613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.087470 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.087479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.088071 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.088075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.088196 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.088214 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:41:35.088276 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:41:35.088345 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":50,"thread_start_us":125,"threads_started":1}
I20260927 18:41:35.089092 24407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1139d41659f9454bb02f30621f2643d0" candidate_uuid: "6d0d953551dd4556914f1c0cb12665b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e03abe93f240dfb347069c17ef7b09" is_pre_election: true
I20260927 18:41:35.089105 24541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1139d41659f9454bb02f30621f2643d0" candidate_uuid: "6d0d953551dd4556914f1c0cb12665b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" is_pre_election: true
I20260927 18:41:35.089360 24407 raft_consensus.cc:2454] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0d953551dd4556914f1c0cb12665b6 in term 0.
I20260927 18:41:35.089370 24541 raft_consensus.cc:2454] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0d953551dd4556914f1c0cb12665b6 in term 0.
I20260927 18:41:35.089813 24611 leader_election.cc:302] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55bb4a3b7d3b458ebcfd9d9c77085bf9, 6d0d953551dd4556914f1c0cb12665b6; no voters:
I20260927 18:41:35.090065 24746 raft_consensus.cc:2781] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:35.090143 24746 raft_consensus.cc:482] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:35.090190 24746 raft_consensus.cc:3037] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:35.091135 24746 raft_consensus.cc:504] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:35.091423 24746 leader_election.cc:288] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [CANDIDATE]: Term 1 election: Requested vote from peers a4e03abe93f240dfb347069c17ef7b09 (127.6.4.1:36225), 55bb4a3b7d3b458ebcfd9d9c77085bf9 (127.6.4.2:41273)
I20260927 18:41:35.091733 24407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1139d41659f9454bb02f30621f2643d0" candidate_uuid: "6d0d953551dd4556914f1c0cb12665b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e03abe93f240dfb347069c17ef7b09"
I20260927 18:41:35.091754 24541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1139d41659f9454bb02f30621f2643d0" candidate_uuid: "6d0d953551dd4556914f1c0cb12665b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9"
I20260927 18:41:35.091848 24541 raft_consensus.cc:3037] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:35.091856 24407 raft_consensus.cc:3037] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:35.092840 24541 raft_consensus.cc:2454] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0d953551dd4556914f1c0cb12665b6 in term 1.
I20260927 18:41:35.092888 24407 raft_consensus.cc:2454] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0d953551dd4556914f1c0cb12665b6 in term 1.
I20260927 18:41:35.093173 24610 leader_election.cc:302] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d0d953551dd4556914f1c0cb12665b6, a4e03abe93f240dfb347069c17ef7b09; no voters:
I20260927 18:41:35.093338 24746 raft_consensus.cc:2781] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:35.093550 24746 raft_consensus.cc:686] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 1 LEADER]: Becoming Leader. State: Replica: 6d0d953551dd4556914f1c0cb12665b6, State: Running, Role: LEADER
I20260927 18:41:35.093883 24746 consensus_queue.cc:237] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } }
I20260927 18:41:35.096776 24261 catalog_manager.cc:5495] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0d953551dd4556914f1c0cb12665b6 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "6d0d953551dd4556914f1c0cb12665b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0d953551dd4556914f1c0cb12665b6" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:35.178694 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.175251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.6.4.3:41231)
0927 18:41:35.175380 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:35.175384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:35.175399 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:41:35.175509 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:35.175511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:35.175568 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:35.175658 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:35.175916 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.176456 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.177272 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.177479 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.177563 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.177873 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:35.178275 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:35.178404 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:35.178510 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 18:41:35.178548 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:41:35.184310 24541 raft_consensus.cc:1261] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 1 FOLLOWER]: Refusing update from remote peer 6d0d953551dd4556914f1c0cb12665b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:35.184310 24407 raft_consensus.cc:1261] T 1139d41659f9454bb02f30621f2643d0 P a4e03abe93f240dfb347069c17ef7b09 [term 1 FOLLOWER]: Refusing update from remote peer 6d0d953551dd4556914f1c0cb12665b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:35.185045 24748 consensus_queue.cc:1035] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4e03abe93f240dfb347069c17ef7b09" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 36225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:35.185212 24746 consensus_queue.cc:1035] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55bb4a3b7d3b458ebcfd9d9c77085bf9" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:35.192868 24752 mvcc.cc:205] Tried to move back new op lower bound from 7334029292267900928 to 7334029291906654208. Current Snapshot: MvccSnapshot[applied={T|T < 7334029292267900928}]
I20260927 18:41:35.195155 24753 mvcc.cc:205] Tried to move back new op lower bound from 7334029292267900928 to 7334029291906654208. Current Snapshot: MvccSnapshot[applied={T|T < 7334029292267900928 or (T in {7334029292267900928})}]
I20260927 18:41:35.199615 24261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:35.199798 24261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:35.202143 24261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b49935e832040459b73f5a96a4bbb46: Sending DeleteTablet for 3 replicas of tablet 1139d41659f9454bb02f30621f2643d0
I20260927 18:41:35.202906 24387 tablet_service.cc:1514] Processing DeleteTablet for tablet 1139d41659f9454bb02f30621f2643d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:35 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:51106
I20260927 18:41:35.202989 24655 tablet_service.cc:1514] Processing DeleteTablet for tablet 1139d41659f9454bb02f30621f2643d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:35 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:48942
I20260927 18:41:35.203001 24521 tablet_service.cc:1514] Processing DeleteTablet for tablet 1139d41659f9454bb02f30621f2643d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:35 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:48630
I20260927 18:41:35.203487 24760 tablet_replica.cc:330] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: stopping tablet replica
I20260927 18:41:35.203591 24762 tablet_replica.cc:330] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: stopping tablet replica
I20260927 18:41:35.203804 24760 raft_consensus.cc:2229] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:35.203958 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24324
I20260927 18:41:35.203969 24762 raft_consensus.cc:2229] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:35.204247 24760 raft_consensus.cc:2258] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:35.204439 24762 raft_consensus.cc:2258] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:35.206437 24762 ts_tablet_manager.cc:1905] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:35.207626 24760 ts_tablet_manager.cc:1905] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:35.209116 24762 ts_tablet_manager.cc:1918] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:35.209211 24762 log.cc:1196] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/1139d41659f9454bb02f30621f2643d0
I20260927 18:41:35.209512 24762 ts_tablet_manager.cc:1939] T 1139d41659f9454bb02f30621f2643d0 P 55bb4a3b7d3b458ebcfd9d9c77085bf9: Deleting consensus metadata
I20260927 18:41:35.210346 24760 ts_tablet_manager.cc:1918] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:35.210486 24760 log.cc:1196] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/1139d41659f9454bb02f30621f2643d0
I20260927 18:41:35.210796 24760 ts_tablet_manager.cc:1939] T 1139d41659f9454bb02f30621f2643d0 P 6d0d953551dd4556914f1c0cb12665b6: Deleting consensus metadata
I20260927 18:41:35.210772 24249 catalog_manager.cc:4841] TS 55bb4a3b7d3b458ebcfd9d9c77085bf9 (127.6.4.2:41273): tablet 1139d41659f9454bb02f30621f2643d0 (table test-table [id=0d715f34784c41cb8ac632aba087218e]) successfully deleted
W20260927 18:41:35.211905 24248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:36225 (error 108)
I20260927 18:41:35.211964 24247 catalog_manager.cc:4841] TS 6d0d953551dd4556914f1c0cb12665b6 (127.6.4.3:41231): tablet 1139d41659f9454bb02f30621f2643d0 (table test-table [id=0d715f34784c41cb8ac632aba087218e]) successfully deleted
I20260927 18:41:35.212004 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24458
I20260927 18:41:35.214409 24734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.214121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:36225 (local address 127.0.0.1:51118)
0927 18:41:35.214255 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:35.214323 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:36225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260927 18:41:35.214617 24248 catalog_manager.cc:4584] TS a4e03abe93f240dfb347069c17ef7b09 (127.6.4.1:36225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1139d41659f9454bb02f30621f2643d0: Network error: Client connection negotiation failed: client connection to 127.6.4.1:36225: connect: Connection refused (error 111)
I20260927 18:41:35.217317 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24592
I20260927 18:41:35.221999 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24230
2026-09-27T18:41:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2460 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T18:41:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:35Z Disabled control of system clock
I20260927 18:41:35.249518 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33887
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:33887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:41:35.360122 24767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:35.360395 24767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:35.360430 24767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:35.364418 24767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:35.364478 24767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:35.364491 24767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:35.364507 24767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:35.364521 24767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:35.369324 24767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:33887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:33887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:35.370324 24767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:35.371616 24767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:35.379750 24772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:35.379812 24767 server_base.cc:1034] running on GCE node
W20260927 18:41:35.379712 24775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.379712 24773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:35.380654 24767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:35.381700 24767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:35.382858 24767 hybrid_clock.cc:648] HybridClock initialized: now 1790534495382843 us; error 25 us; skew 500 ppm
I20260927 18:41:35.384654 24767 webserver.cc:460] Webserver started at http://127.6.4.62:36051/ using document root <none> and password file <none>
I20260927 18:41:35.385188 24767 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:35.385245 24767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:35.385440 24767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:35.387428 24767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "e8188fe6a9c84ad28b66bf13ba929ec0"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "69d8a80ea7b6b5c24568089213cfb5d5"
server_key_iv: "18e9367a424eaf8637853f8e460d5ec2"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.387941 24767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "e8188fe6a9c84ad28b66bf13ba929ec0"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "69d8a80ea7b6b5c24568089213cfb5d5"
server_key_iv: "18e9367a424eaf8637853f8e460d5ec2"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.391458 24767 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:41:35.393791 24781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.394971 24767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260927 18:41:35.395102 24767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "e8188fe6a9c84ad28b66bf13ba929ec0"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "69d8a80ea7b6b5c24568089213cfb5d5"
server_key_iv: "18e9367a424eaf8637853f8e460d5ec2"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.395195 24767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:35.417325 24767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:35.418040 24767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:35.418212 24767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:35.424806 24767 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:33887
I20260927 18:41:35.424840 24833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:33887 every 8 connection(s)
I20260927 18:41:35.425696 24767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:35.428974 24834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:35.434314 24834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0: Bootstrap starting.
I20260927 18:41:35.434907 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24767
I20260927 18:41:35.435034 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:35.435319 6160 external_mini_cluster.cc:1428] Setting key 43f282248d9c9fe86f4222b839e59fff
I20260927 18:41:35.437669 24834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:35.438385 24834 log.cc:824] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:35.440138 24834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0: No bootstrap required, opened a new log
I20260927 18:41:35.443478 24834 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33887 } }
I20260927 18:41:35.443710 24834 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:35.443756 24834 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8188fe6a9c84ad28b66bf13ba929ec0, State: Initialized, Role: FOLLOWER
I20260927 18:41:35.444190 24834 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33887 } }
I20260927 18:41:35.444317 24834 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:35.444378 24834 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:35.444430 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.436705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.6.4.62:33887)
0927 18:41:35.437321 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:35.437336 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:35.437371 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:41:35.438069 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:35.438088 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:35.438514 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:35.438837 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:35.439243 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.440275 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.441135 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.441657 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.441834 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.442578 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:35.442582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:35.442594 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:35.442978 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:35.443033 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:35.443053 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:35.443137 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:35.443275 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 18:41:35.443526 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":104,"threads_started":1}
I20260927 18:41:35.444465 24834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:35.445308 24834 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33887 } }
I20260927 18:41:35.445648 24834 leader_election.cc:302] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8188fe6a9c84ad28b66bf13ba929ec0; no voters:
I20260927 18:41:35.445992 24834 leader_election.cc:288] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:35.446076 24839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:35.446306 24839 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [term 1 LEADER]: Becoming Leader. State: Replica: e8188fe6a9c84ad28b66bf13ba929ec0, State: Running, Role: LEADER
I20260927 18:41:35.446619 24839 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33887 } }
I20260927 18:41:35.447162 24834 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:35.447715 24841 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8188fe6a9c84ad28b66bf13ba929ec0. Latest consensus state: current_term: 1 leader_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33887 } } }
I20260927 18:41:35.447849 24841 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:35.448145 24840 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8188fe6a9c84ad28b66bf13ba929ec0" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 33887 } } }
I20260927 18:41:35.448244 24840 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:35.448447 24846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:35.451444 24846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:35.457242 24846 catalog_manager.cc:1347] Generated new cluster ID: f440e7d7b4e8456e8c889ff6812e54a7
I20260927 18:41:35.457314 24846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:35.463788 24846 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:35.465016 24846 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:35.477257 24846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0: Generated new TSK 0
I20260927 18:41:35.478102 24846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:35.485627 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33887
--builtin_ntp_servers=127.6.4.20:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:41:35.593292 24858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:35.593557 24858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:35.593612 24858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:35.597595 24858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:35.597652 24858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:35.597728 24858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:35.602527 24858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:35.603579 24858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:35.604974 24858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:35.614143 24864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.614162 24866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.614231 24863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:35.614312 24858 server_base.cc:1034] running on GCE node
I20260927 18:41:35.614830 24858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:35.615450 24858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:35.616598 24858 hybrid_clock.cc:648] HybridClock initialized: now 1790534495616581 us; error 29 us; skew 500 ppm
I20260927 18:41:35.618373 24858 webserver.cc:460] Webserver started at http://127.6.4.1:44395/ using document root <none> and password file <none>
I20260927 18:41:35.618975 24858 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:35.619041 24858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:35.619227 24858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:35.621240 24858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "59eb3a98461e41b99b5a256ee88cb76b"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "e4930d969ea7b20a11728ad243c30f37"
server_key_iv: "d892305eb5fd6169219056753b2523f1"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.621788 24858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "59eb3a98461e41b99b5a256ee88cb76b"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "e4930d969ea7b20a11728ad243c30f37"
server_key_iv: "d892305eb5fd6169219056753b2523f1"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.625262 24858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:35.627532 24872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.628741 24858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:35.628844 24858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "59eb3a98461e41b99b5a256ee88cb76b"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "e4930d969ea7b20a11728ad243c30f37"
server_key_iv: "d892305eb5fd6169219056753b2523f1"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.628940 24858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:35.642429 24858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:35.643122 24858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:35.643293 24858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:35.643867 24858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:35.644834 24858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:35.644881 24858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.644933 24858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:35.644961 24858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.653267 24858 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:44857
I20260927 18:41:35.653301 24985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:44857 every 8 connection(s)
I20260927 18:41:35.654183 24858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:35.661585 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24858
I20260927 18:41:35.661705 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:35.661968 6160 external_mini_cluster.cc:1428] Setting key ceb927bcb48d98203b58a0f869e9251d
I20260927 18:41:35.663501 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.656010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:33375 (local address 127.6.4.62:33887)
0927 18:41:35.656196 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:35.656200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:35.657248 (+ 1048us) server_negotiation.cc:407] Connection header received
0927 18:41:35.658554 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:35.658557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:35.658599 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:35.658670 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:35.659515 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.660047 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.661184 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.661435 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.661532 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.662614 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:35.662618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:35.662621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:35.662633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:35.662655 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:35.662664 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:35.662732 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:35.663311 (+ 579us) server_negotiation.cc:299] Negotiation successful
0927 18:41:35.663366 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 18:41:35.663942 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33887
--builtin_ntp_servers=127.6.4.20:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:41:35.664438 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.656427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33887 (local address 127.6.4.1:33375)
0927 18:41:35.657101 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:35.657142 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:35.658373 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:35.658830 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:35.658836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:35.658921 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:35.659407 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.659419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.660204 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.660207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.661092 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.661101 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.661531 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.661579 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.661732 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.662337 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:35.662339 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:35.662391 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:35.662473 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:35.663155 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:35.663165 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:35.663171 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:35.663308 (+ 137us) client_negotiation.cc:240] Negotiation successful
0927 18:41:35.663563 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":111,"threads_started":1}
I20260927 18:41:35.665642 24986 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33887
I20260927 18:41:35.665901 24986 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:35.666428 24986 heartbeater.cc:511] Master 127.6.4.62:33887 requested a full tablet report, sending...
I20260927 18:41:35.667914 24798 ts_manager.cc:194] Registered new tserver with Master: 59eb3a98461e41b99b5a256ee88cb76b (127.6.4.1:44857)
I20260927 18:41:35.669528 24798 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.1:33375
W20260927 18:41:35.772845 24989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:35.773099 24989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:35.773141 24989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:35.777130 24989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:35.777181 24989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:35.777241 24989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:35.781890 24989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:35.782897 24989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:35.784227 24989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:35.793183 24995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.793190 24997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.793202 24994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:35.793494 24989 server_base.cc:1034] running on GCE node
I20260927 18:41:35.793885 24989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:35.794476 24989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:35.795626 24989 hybrid_clock.cc:648] HybridClock initialized: now 1790534495795603 us; error 31 us; skew 500 ppm
I20260927 18:41:35.797375 24989 webserver.cc:460] Webserver started at http://127.6.4.2:43883/ using document root <none> and password file <none>
I20260927 18:41:35.797899 24989 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:35.797964 24989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:35.798171 24989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:35.800220 24989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "ca4dace97d774290bda9424cdb85e940"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "f1a3488beaeab0dbc0d9c633bd62ea1b"
server_key_iv: "95fc9e36120b6cb4d4113943819816e6"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.800729 24989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "ca4dace97d774290bda9424cdb85e940"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "f1a3488beaeab0dbc0d9c633bd62ea1b"
server_key_iv: "95fc9e36120b6cb4d4113943819816e6"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.804090 24989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:41:35.806442 25003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.807608 24989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:35.807701 24989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "ca4dace97d774290bda9424cdb85e940"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "f1a3488beaeab0dbc0d9c633bd62ea1b"
server_key_iv: "95fc9e36120b6cb4d4113943819816e6"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.807802 24989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:35.832566 24989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:35.833264 24989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:35.833413 24989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:35.833961 24989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:35.834825 24989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:35.834869 24989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.834905 24989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:35.834932 24989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.842887 24989 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:46035
I20260927 18:41:35.842926 25116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:46035 every 8 connection(s)
I20260927 18:41:35.843741 24989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:35.850885 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 24989
I20260927 18:41:35.851001 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:35.851267 6160 external_mini_cluster.cc:1428] Setting key db8962a1c0c09af1eaf3ec199748c031
I20260927 18:41:35.852567 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.845483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:46513 (local address 127.6.4.62:33887)
0927 18:41:35.845632 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:35.845636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:35.846619 (+ 983us) server_negotiation.cc:407] Connection header received
0927 18:41:35.848015 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:35.848019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:35.848076 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:35.848174 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:35.849075 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.849596 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.850722 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.850939 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.851025 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.852134 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:35.852138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:35.852140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:35.852152 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:35.852175 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:35.852184 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:35.852253 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:35.852374 (+ 121us) server_negotiation.cc:299] Negotiation successful
0927 18:41:35.852432 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:41:35.853214 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33887
--builtin_ntp_servers=127.6.4.20:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:41:35.853506 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:35.845852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33887 (local address 127.6.4.2:46513)
0927 18:41:35.846479 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:35.846517 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:35.847814 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:35.848349 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:35.848356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:35.848440 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:35.848939 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.848951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.849717 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.849720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.850602 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:35.850611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:35.851028 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:35.851086 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:35.851240 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:35.851851 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:35.851853 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:35.851911 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:35.851977 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:35.852255 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:35.852261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:35.852265 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:35.852355 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 18:41:35.852627 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260927 18:41:35.854622 25117 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33887
I20260927 18:41:35.854836 25117 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:35.855319 25117 heartbeater.cc:511] Master 127.6.4.62:33887 requested a full tablet report, sending...
I20260927 18:41:35.856323 24798 ts_manager.cc:194] Registered new tserver with Master: ca4dace97d774290bda9424cdb85e940 (127.6.4.2:46035)
I20260927 18:41:35.857028 24798 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.2:46513
W20260927 18:41:35.959992 25120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:35.960271 25120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:35.960319 25120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:35.965284 25120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:35.965348 25120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:35.965404 25120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:35.970198 25120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:33887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:35.971387 25120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:35.972759 25120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:35.982075 25126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.982050 25125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:35.982095 25128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:35.982275 25120 server_base.cc:1034] running on GCE node
I20260927 18:41:35.982631 25120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:35.983239 25120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:35.984390 25120 hybrid_clock.cc:648] HybridClock initialized: now 1790534495984372 us; error 26 us; skew 500 ppm
I20260927 18:41:35.986155 25120 webserver.cc:460] Webserver started at http://127.6.4.3:35793/ using document root <none> and password file <none>
I20260927 18:41:35.986686 25120 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:35.986750 25120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:35.986943 25120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:35.989013 25120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "61febf9fd4ed42b98b20259b00b986a8"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "937505498da1d087ccc38ac9c8465da5"
server_key_iv: "07193443af265cd2bb52db59d94f3c8e"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.989537 25120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "61febf9fd4ed42b98b20259b00b986a8"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "937505498da1d087ccc38ac9c8465da5"
server_key_iv: "07193443af265cd2bb52db59d94f3c8e"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.992992 25120 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:35.995255 25134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:35.996418 25120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:35.996521 25120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "61febf9fd4ed42b98b20259b00b986a8"
format_stamp: "Formatted at 2026-09-27 18:41:35 on dist-test-slave-bzzj"
server_key: "937505498da1d087ccc38ac9c8465da5"
server_key_iv: "07193443af265cd2bb52db59d94f3c8e"
server_key_version: "encryptionkey@0"
I20260927 18:41:35.996614 25120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:36.007753 25120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:36.008456 25120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:36.008620 25120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:36.009214 25120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:36.010149 25120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:36.010193 25120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:36.010236 25120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:36.010263 25120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:36.018201 25120 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:37137
I20260927 18:41:36.018229 25247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:37137 every 8 connection(s)
I20260927 18:41:36.019061 25120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:36.019361 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25120
I20260927 18:41:36.019461 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:36.019761 6160 external_mini_cluster.cc:1428] Setting key b95f2f63a78bfaade6e9a0e3e26c778f
I20260927 18:41:36.028200 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.020853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:43525 (local address 127.6.4.62:33887)
0927 18:41:36.021053 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.021057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.022339 (+ 1282us) server_negotiation.cc:407] Connection header received
0927 18:41:36.023735 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.023738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.023793 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.023862 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.024778 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.025312 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.026419 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.026638 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.026721 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.027701 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.027705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.027708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:36.027722 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.027740 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:36.027748 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.027811 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.027946 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.028006 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260927 18:41:36.029017 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.021461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:33887 (local address 127.6.4.3:43525)
0927 18:41:36.022197 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.022248 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.023539 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.024025 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.024031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.024164 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:36.024645 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.024659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.025426 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.025429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.026308 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.026317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.026717 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.026783 (+ 66us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.026933 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.027456 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:36.027458 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:36.027498 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:36.027562 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:36.027822 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:36.027832 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:36.027838 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.027942 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.028245 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":105,"threads_started":1}
I20260927 18:41:36.030068 25248 heartbeater.cc:347] Connected to a master server at 127.6.4.62:33887
I20260927 18:41:36.030282 25248 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:36.030777 25248 heartbeater.cc:511] Master 127.6.4.62:33887 requested a full tablet report, sending...
I20260927 18:41:36.031715 24798 ts_manager.cc:194] Registered new tserver with Master: 61febf9fd4ed42b98b20259b00b986a8 (127.6.4.3:37137)
I20260927 18:41:36.032356 24798 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.3:43525
I20260927 18:41:36.033509 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:36.038594 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.035315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.6.4.62:33887)
0927 18:41:36.035442 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.035446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.035537 (+ 91us) server_negotiation.cc:407] Connection header received
0927 18:41:36.035632 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.035634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.035673 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.035732 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:36.035950 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.036485 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.037295 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.037493 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.037557 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.038232 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:36.038235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:36.038237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:36.038247 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:36.038263 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:36.038270 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:36.038354 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.038431 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.038460 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:41:36.041137 24798 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:36.043963 24798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:36.057017 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.051680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:37137 (local address 127.0.0.1:37314)
0927 18:41:36.052003 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.052045 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.052183 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.052887 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.052893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.052911 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:36.053013 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.053017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.054265 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.054268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.055427 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.055437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.056492 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.056495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.056602 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.056640 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.056742 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.056863 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":83,"threads_started":1}
I20260927 18:41:36.057057 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.051825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.6.4.3:37137)
0927 18:41:36.052166 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.052176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.052198 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:41:36.052319 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.052325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.052531 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.052681 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:36.053131 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.054165 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.055562 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.056379 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.056481 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.056666 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.056784 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.056878 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":133,"threads_started":1}
I20260927 18:41:36.057463 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.052379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:46035 (local address 127.0.0.1:59376)
0927 18:41:36.052717 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.052729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.052804 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.053705 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.053708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.053726 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:36.053832 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.053838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.054994 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.054997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.056177 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.056186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.057127 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.057131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.057224 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.057245 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.057287 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.057344 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":100,"threads_started":1}
I20260927 18:41:36.058104 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.052551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.6.4.2:46035)
0927 18:41:36.053351 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.053363 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.053384 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:41:36.053452 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.053456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.053589 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.053696 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:36.053931 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.054905 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.056297 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.056998 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.057621 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.057822 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.057870 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.057958 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":103,"threads_started":1}
I20260927 18:41:36.058619 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.052670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44857 (local address 127.0.0.1:40678)
0927 18:41:36.054330 (+ 1660us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.054340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.054417 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.054764 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.054767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.054782 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:36.054859 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.054865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.056235 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.056238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.057274 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.057284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.058091 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.058094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.058322 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.058343 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.058406 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.058480 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1609,"thread_start_us":82,"threads_started":1}
I20260927 18:41:36.059072 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.053169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.6.4.1:44857)
0927 18:41:36.053551 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.053566 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.054425 (+ 859us) server_negotiation.cc:407] Connection header received
0927 18:41:36.054505 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.054510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.054645 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.054765 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:36.054959 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.056131 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.057388 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.057986 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.058079 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.058248 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.058831 (+ 583us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.058927 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":102,"threads_started":1}
I20260927 18:41:36.060631 25182 tablet_service.cc:1467] Processing CreateTablet for tablet 2367afe2765745618bad5a0568980e6c (DEFAULT_TABLE table=test-table [id=deae32d431194c4098fb3cacdc547097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:36.060663 25051 tablet_service.cc:1467] Processing CreateTablet for tablet 2367afe2765745618bad5a0568980e6c (DEFAULT_TABLE table=test-table [id=deae32d431194c4098fb3cacdc547097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:36.061579 25182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2367afe2765745618bad5a0568980e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:36.061663 25051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2367afe2765745618bad5a0568980e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:36.061877 24920 tablet_service.cc:1467] Processing CreateTablet for tablet 2367afe2765745618bad5a0568980e6c (DEFAULT_TABLE table=test-table [id=deae32d431194c4098fb3cacdc547097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:36.062808 24920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2367afe2765745618bad5a0568980e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:36.067967 25265 tablet_bootstrap.cc:492] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Bootstrap starting.
I20260927 18:41:36.068951 25266 tablet_bootstrap.cc:492] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Bootstrap starting.
I20260927 18:41:36.069867 25267 tablet_bootstrap.cc:492] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: Bootstrap starting.
I20260927 18:41:36.070102 25265 tablet_bootstrap.cc:654] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:36.070799 25265 log.cc:824] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:36.070989 25266 tablet_bootstrap.cc:654] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:36.071760 25267 tablet_bootstrap.cc:654] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:36.071786 25266 log.cc:824] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:36.072480 25267 log.cc:824] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:36.072597 25265 tablet_bootstrap.cc:492] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: No bootstrap required, opened a new log
I20260927 18:41:36.072772 25265 ts_tablet_manager.cc:1397] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:41:36.073590 25266 tablet_bootstrap.cc:492] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: No bootstrap required, opened a new log
I20260927 18:41:36.073782 25266 ts_tablet_manager.cc:1397] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260927 18:41:36.074357 25267 tablet_bootstrap.cc:492] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: No bootstrap required, opened a new log
I20260927 18:41:36.074520 25267 ts_tablet_manager.cc:1397] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 18:41:36.076251 25265 raft_consensus.cc:348] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.076484 25265 raft_consensus.cc:374] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:36.076536 25265 raft_consensus.cc:729] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61febf9fd4ed42b98b20259b00b986a8, State: Initialized, Role: FOLLOWER
I20260927 18:41:36.076975 25265 consensus_queue.cc:260] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.077625 25248 heartbeater.cc:503] Master 127.6.4.62:33887 was elected leader, sending a full tablet report...
I20260927 18:41:36.078011 25265 ts_tablet_manager.cc:1428] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:41:36.077925 25266 raft_consensus.cc:348] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.078184 25266 raft_consensus.cc:374] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:36.078234 25266 raft_consensus.cc:729] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4dace97d774290bda9424cdb85e940, State: Initialized, Role: FOLLOWER
I20260927 18:41:36.078554 25267 raft_consensus.cc:348] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.078701 25266 consensus_queue.cc:260] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.078797 25267 raft_consensus.cc:374] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:36.078842 25267 raft_consensus.cc:729] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59eb3a98461e41b99b5a256ee88cb76b, State: Initialized, Role: FOLLOWER
I20260927 18:41:36.079296 25267 consensus_queue.cc:260] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.079730 25266 ts_tablet_manager.cc:1428] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260927 18:41:36.080030 24986 heartbeater.cc:503] Master 127.6.4.62:33887 was elected leader, sending a full tablet report...
I20260927 18:41:36.080080 25117 heartbeater.cc:503] Master 127.6.4.62:33887 was elected leader, sending a full tablet report...
I20260927 18:41:36.080291 25267 ts_tablet_manager.cc:1428] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260927 18:41:36.094663 25118 tablet.cc:2369] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:41:36.155300 24987 tablet.cc:2369] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:36.164602 25271 raft_consensus.cc:482] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:36.164805 25271 raft_consensus.cc:504] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.165930 25271 leader_election.cc:288] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca4dace97d774290bda9424cdb85e940 (127.6.4.2:46035), 59eb3a98461e41b99b5a256ee88cb76b (127.6.4.1:44857)
I20260927 18:41:36.169736 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.166166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:46035 (local address 127.6.4.3:44261)
0927 18:41:36.166299 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.166313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.166420 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.166642 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.166644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.166660 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:36.166776 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.166783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.167637 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.167641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.168603 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.168612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.169313 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.169317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.169421 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.169447 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.169535 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.169606 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:41:36.169736 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.166308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:44261 (local address 127.6.4.2:46035)
0927 18:41:36.166439 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.166444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.166462 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:41:36.166516 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.166519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.166563 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.166624 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:36.166911 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.167540 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.168719 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.169199 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.169274 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.169323 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.169543 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.169603 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:41:36.170054 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.166810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:40727 (local address 127.6.4.1:44857)
0927 18:41:36.166933 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.166937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.166950 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:41:36.166991 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.166993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.167030 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.167084 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:36.167356 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.167959 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.169043 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.169519 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.169609 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.169660 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.169870 (+ 210us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.169946 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:41:36.170336 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.166423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44857 (local address 127.6.4.3:40727)
0927 18:41:36.166730 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.166743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:36.166823 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:36.167116 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:36.167119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:36.167136 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:36.167254 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.167260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.168064 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.168066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.168902 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:36.168911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.169628 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.169631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:36.169722 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.169744 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:41:36.170178 (+ 434us) client_negotiation.cc:240] Negotiation successful
0927 18:41:36.170235 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260927 18:41:36.170434 25071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2367afe2765745618bad5a0568980e6c" candidate_uuid: "61febf9fd4ed42b98b20259b00b986a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4dace97d774290bda9424cdb85e940" is_pre_election: true
I20260927 18:41:36.170763 25071 raft_consensus.cc:2454] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61febf9fd4ed42b98b20259b00b986a8 in term 0.
I20260927 18:41:36.170881 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2367afe2765745618bad5a0568980e6c" candidate_uuid: "61febf9fd4ed42b98b20259b00b986a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59eb3a98461e41b99b5a256ee88cb76b" is_pre_election: true
I20260927 18:41:36.171141 24940 raft_consensus.cc:2454] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61febf9fd4ed42b98b20259b00b986a8 in term 0.
I20260927 18:41:36.171283 25137 leader_election.cc:302] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61febf9fd4ed42b98b20259b00b986a8, ca4dace97d774290bda9424cdb85e940; no voters:
I20260927 18:41:36.171540 25271 raft_consensus.cc:2781] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:36.171612 25271 raft_consensus.cc:482] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:36.171641 25271 raft_consensus.cc:3037] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:36.172768 25271 raft_consensus.cc:504] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.173089 25271 leader_election.cc:288] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [CANDIDATE]: Term 1 election: Requested vote from peers ca4dace97d774290bda9424cdb85e940 (127.6.4.2:46035), 59eb3a98461e41b99b5a256ee88cb76b (127.6.4.1:44857)
I20260927 18:41:36.173390 25071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2367afe2765745618bad5a0568980e6c" candidate_uuid: "61febf9fd4ed42b98b20259b00b986a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4dace97d774290bda9424cdb85e940"
I20260927 18:41:36.173449 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2367afe2765745618bad5a0568980e6c" candidate_uuid: "61febf9fd4ed42b98b20259b00b986a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59eb3a98461e41b99b5a256ee88cb76b"
I20260927 18:41:36.173497 25071 raft_consensus.cc:3037] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:36.173573 24940 raft_consensus.cc:3037] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:36.174477 25071 raft_consensus.cc:2454] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61febf9fd4ed42b98b20259b00b986a8 in term 1.
I20260927 18:41:36.174564 24940 raft_consensus.cc:2454] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61febf9fd4ed42b98b20259b00b986a8 in term 1.
I20260927 18:41:36.174734 25137 leader_election.cc:302] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61febf9fd4ed42b98b20259b00b986a8, ca4dace97d774290bda9424cdb85e940; no voters:
I20260927 18:41:36.174883 25271 raft_consensus.cc:2781] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:36.175148 25271 raft_consensus.cc:686] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 1 LEADER]: Becoming Leader. State: Replica: 61febf9fd4ed42b98b20259b00b986a8, State: Running, Role: LEADER
I20260927 18:41:36.175503 25271 consensus_queue.cc:237] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } }
I20260927 18:41:36.178679 24797 catalog_manager.cc:5495] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61febf9fd4ed42b98b20259b00b986a8 (127.6.4.3). New cstate: current_term: 1 leader_uuid: "61febf9fd4ed42b98b20259b00b986a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61febf9fd4ed42b98b20259b00b986a8" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 37137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:36.210464 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.207037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.6.4.3:37137)
0927 18:41:36.207194 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:36.207199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:36.207212 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:41:36.207286 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:36.207288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:36.207342 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:36.207431 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:36.207672 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.208188 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.209043 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:36.209254 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:36.209343 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:36.209644 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:36.210043 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:36.210170 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:36.210292 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 18:41:36.210324 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 18:41:36.216157 25071 raft_consensus.cc:1261] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 1 FOLLOWER]: Refusing update from remote peer 61febf9fd4ed42b98b20259b00b986a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:36.216176 24940 raft_consensus.cc:1261] T 2367afe2765745618bad5a0568980e6c P 59eb3a98461e41b99b5a256ee88cb76b [term 1 FOLLOWER]: Refusing update from remote peer 61febf9fd4ed42b98b20259b00b986a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:36.216809 25275 consensus_queue.cc:1035] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca4dace97d774290bda9424cdb85e940" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 46035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:36.216989 25271 consensus_queue.cc:1035] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59eb3a98461e41b99b5a256ee88cb76b" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:36.232290 24797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:36.232517 24797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:36.235028 24797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8188fe6a9c84ad28b66bf13ba929ec0: Sending DeleteTablet for 3 replicas of tablet 2367afe2765745618bad5a0568980e6c
I20260927 18:41:36.235906 25182 tablet_service.cc:1514] Processing DeleteTablet for tablet 2367afe2765745618bad5a0568980e6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:36 UTC) from {username='slave'} at 127.0.0.1:37314
I20260927 18:41:36.236164 24920 tablet_service.cc:1514] Processing DeleteTablet for tablet 2367afe2765745618bad5a0568980e6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:36 UTC) from {username='slave'} at 127.0.0.1:40678
I20260927 18:41:36.236441 25051 tablet_service.cc:1514] Processing DeleteTablet for tablet 2367afe2765745618bad5a0568980e6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:36 UTC) from {username='slave'} at 127.0.0.1:59376
I20260927 18:41:36.236544 25288 tablet_replica.cc:330] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: stopping tablet replica
I20260927 18:41:36.236865 25290 tablet_replica.cc:330] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: stopping tablet replica
I20260927 18:41:36.236884 25288 raft_consensus.cc:2229] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:36.237231 25290 raft_consensus.cc:2229] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:36.237249 25288 raft_consensus.cc:2258] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:36.237289 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24858
I20260927 18:41:36.237645 25290 raft_consensus.cc:2258] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:36.239162 25290 ts_tablet_manager.cc:1905] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:36.239231 25288 ts_tablet_manager.cc:1905] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:36.242002 25290 ts_tablet_manager.cc:1918] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:36.242048 25288 ts_tablet_manager.cc:1918] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:36.242137 25288 log.cc:1196] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/2367afe2765745618bad5a0568980e6c
I20260927 18:41:36.242238 25290 log.cc:1196] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/2367afe2765745618bad5a0568980e6c
I20260927 18:41:36.242431 25288 ts_tablet_manager.cc:1939] T 2367afe2765745618bad5a0568980e6c P 61febf9fd4ed42b98b20259b00b986a8: Deleting consensus metadata
I20260927 18:41:36.242645 25290 ts_tablet_manager.cc:1939] T 2367afe2765745618bad5a0568980e6c P ca4dace97d774290bda9424cdb85e940: Deleting consensus metadata
I20260927 18:41:36.243636 24784 catalog_manager.cc:4841] TS 61febf9fd4ed42b98b20259b00b986a8 (127.6.4.3:37137): tablet 2367afe2765745618bad5a0568980e6c (table test-table [id=deae32d431194c4098fb3cacdc547097]) successfully deleted
I20260927 18:41:36.243866 24784 catalog_manager.cc:4841] TS ca4dace97d774290bda9424cdb85e940 (127.6.4.2:46035): tablet 2367afe2765745618bad5a0568980e6c (table test-table [id=deae32d431194c4098fb3cacdc547097]) successfully deleted
W20260927 18:41:36.245935 24784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:44857 (error 108)
I20260927 18:41:36.246582 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24989
I20260927 18:41:36.246971 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:36.246657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:44857 (local address 127.0.0.1:40692)
0927 18:41:36.246828 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:36.246889 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:44857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260927 18:41:36.247438 24784 catalog_manager.cc:4584] TS 59eb3a98461e41b99b5a256ee88cb76b (127.6.4.1:44857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2367afe2765745618bad5a0568980e6c: Network error: Client connection negotiation failed: client connection to 127.6.4.1:44857: connect: Connection refused (error 111)
I20260927 18:41:36.252120 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25120
I20260927 18:41:36.256914 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 24767
2026-09-27T18:41:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1034 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:36 dist-test-slave-bzzj krb5kdc[25294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:36 dist-test-slave-bzzj krb5kdc[25294](info): set up 2 sockets
Sep 27 18:41:36 dist-test-slave-bzzj krb5kdc[25294](info): commencing operation
krb5kdc: starting...
W20260927 18:41:37.215803 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:41:37.330708 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:36019
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:36019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:37.435916 25307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:37.436201 25307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:37.436255 25307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:37.440163 25307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:37.440217 25307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:37.440241 25307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:37.440261 25307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:37.440281 25307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:37.445416 25307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:36019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:36019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:37.446437 25307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:37.447424 25307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:37.455569 25312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:37.455587 25315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:37.455698 25307 server_base.cc:1034] running on GCE node
W20260927 18:41:37.455587 25313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:37.456426 25307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:37.457518 25307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:37.458681 25307 hybrid_clock.cc:648] HybridClock initialized: now 1790534497458662 us; error 30 us; skew 500 ppm
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:37.461314 25307 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:37.462149 25307 webserver.cc:460] Webserver started at http://127.6.4.62:36803/ using document root <none> and password file <none>
I20260927 18:41:37.462678 25307 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:37.462741 25307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:37.462921 25307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:37.465014 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "f90fac210f8f487992f37f37b488b937"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "5f33753767d84bd122068282aa2ba2af"
server_key_iv: "3e882bd3717f3de296e2ae02515a8080"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.465541 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "f90fac210f8f487992f37f37b488b937"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "5f33753767d84bd122068282aa2ba2af"
server_key_iv: "3e882bd3717f3de296e2ae02515a8080"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.469062 25307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:37.471366 25322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.472580 25307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:37.472693 25307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "f90fac210f8f487992f37f37b488b937"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "5f33753767d84bd122068282aa2ba2af"
server_key_iv: "3e882bd3717f3de296e2ae02515a8080"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.472800 25307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:37.495786 25307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:37.498858 25307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:37.499033 25307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:37.505505 25307 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:36019
I20260927 18:41:37.505554 25374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:36019 every 8 connection(s)
I20260927 18:41:37.506428 25307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:37.509310 25375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:37.514829 25375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937: Bootstrap starting.
I20260927 18:41:37.516350 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25307
I20260927 18:41:37.516474 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:37.516716 6160 external_mini_cluster.cc:1428] Setting key 75195f1d4df261fb082ca8a880018885
I20260927 18:41:37.517325 25375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:37.518056 25375 log.cc:824] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:37.519796 25375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937: No bootstrap required, opened a new log
I20260927 18:41:37.523092 25375 raft_consensus.cc:348] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90fac210f8f487992f37f37b488b937" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36019 } }
I20260927 18:41:37.523301 25375 raft_consensus.cc:374] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:37.523346 25375 raft_consensus.cc:729] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90fac210f8f487992f37f37b488b937, State: Initialized, Role: FOLLOWER
I20260927 18:41:37.523761 25375 consensus_queue.cc:260] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90fac210f8f487992f37f37b488b937" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36019 } }
I20260927 18:41:37.523895 25375 raft_consensus.cc:388] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:37.523958 25375 raft_consensus.cc:482] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:37.524050 25375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:37.525097 25375 raft_consensus.cc:504] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90fac210f8f487992f37f37b488b937" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36019 } }
I20260927 18:41:37.525404 25375 leader_election.cc:302] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f90fac210f8f487992f37f37b488b937; no voters:
I20260927 18:41:37.525703 25375 leader_election.cc:288] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:37.525769 25380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:37.525950 25380 raft_consensus.cc:686] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [term 1 LEADER]: Becoming Leader. State: Replica: f90fac210f8f487992f37f37b488b937, State: Running, Role: LEADER
I20260927 18:41:37.526240 25380 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90fac210f8f487992f37f37b488b937" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36019 } }
I20260927 18:41:37.526947 25375 sys_catalog.cc:564] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:37.527770 25382 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f90fac210f8f487992f37f37b488b937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90fac210f8f487992f37f37b488b937" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36019 } } }
I20260927 18:41:37.527896 25382 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:37.527958 25381 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90fac210f8f487992f37f37b488b937. Latest consensus state: current_term: 1 leader_uuid: "f90fac210f8f487992f37f37b488b937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90fac210f8f487992f37f37b488b937" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 36019 } } }
I20260927 18:41:37.528053 25381 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:37.528281 25389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:37.530866 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:37.517757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.6.4.62:36019)
0927 18:41:37.518155 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:37.518173 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:37.518354 (+ 181us) server_negotiation.cc:407] Connection header received
0927 18:41:37.519080 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:37.519098 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:37.519416 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:37.519719 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:37.520514 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.521506 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.522319 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.522780 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.522917 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:37.525483 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:37.525488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:37.525501 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:37.525531 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:37.527356 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:37.527868 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:37.527874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:37.527879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:37.528333 (+ 454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:37.528659 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:37.528662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:37.528664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:37.529168 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:37.529377 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:37.529616 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 18:41:37.529921 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
I20260927 18:41:37.531359 25389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:37.537175 25389 catalog_manager.cc:1347] Generated new cluster ID: d9e67fe089954499bb704498a4769152
I20260927 18:41:37.537228 25389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:37.549556 25389 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:37.550612 25389 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:37.561439 25389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937: Generated new TSK 0
I20260927 18:41:37.562250 25389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.4.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
Entry for principal kudu/127.6.4.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab.
I20260927 18:41:37.592239 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36019
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:37.698721 25401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:37.698975 25401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:37.699016 25401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:37.702970 25401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:37.703027 25401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:37.703086 25401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:37.708297 25401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.1.keytab
--principal=kudu/127.6.4.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:37.709326 25401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:37.710317 25401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:37.719364 25407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:37.719377 25409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:37.719372 25406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:37.719589 25401 server_base.cc:1034] running on GCE node
I20260927 18:41:37.720054 25401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:37.720675 25401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:37.721839 25401 hybrid_clock.cc:648] HybridClock initialized: now 1790534497721813 us; error 37 us; skew 500 ppm
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:37.724474 25401 init.cc:377] Logged in from keytab as kudu/127.6.4.1@KRBTEST.COM (short username kudu)
I20260927 18:41:37.725283 25401 webserver.cc:460] Webserver started at http://127.6.4.1:33083/ using document root <none> and password file <none>
I20260927 18:41:37.725816 25401 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:37.725879 25401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:37.726092 25401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:37.728358 25401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "f281b98522a448b3a36a838831bd07b7"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "7bfb022ba29eeb5790f4c9cc79d67dbd"
server_key_iv: "eb5d762976f2330bd0592b346ff97172"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.728888 25401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "f281b98522a448b3a36a838831bd07b7"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "7bfb022ba29eeb5790f4c9cc79d67dbd"
server_key_iv: "eb5d762976f2330bd0592b346ff97172"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.732367 25401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 18:41:37.734650 25416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.735924 25401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:37.736035 25401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "f281b98522a448b3a36a838831bd07b7"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "7bfb022ba29eeb5790f4c9cc79d67dbd"
server_key_iv: "eb5d762976f2330bd0592b346ff97172"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.736164 25401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:37.748008 25401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:37.750937 25401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:37.751101 25401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:37.751647 25401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:37.752454 25401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:37.752501 25401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.752550 25401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:37.752578 25401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.760586 25401 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:33737
I20260927 18:41:37.760612 25529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:33737 every 8 connection(s)
I20260927 18:41:37.761492 25401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:37.767971 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25401
I20260927 18:41:37.768122 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:37.768438 6160 external_mini_cluster.cc:1428] Setting key 51d1280188b4c17dbadee3e653fc5797
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.1@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:37.774698 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:37.763206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:59619 (local address 127.6.4.62:36019)
0927 18:41:37.763349 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:37.763352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:37.764206 (+ 854us) server_negotiation.cc:407] Connection header received
0927 18:41:37.765402 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:37.765405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:37.765459 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:37.765534 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:37.766639 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.767172 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.768551 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.768783 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.768880 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:37.771328 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:37.771331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:37.771333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:37.771355 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:37.772640 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:37.773178 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:37.773180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:37.773181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:37.773279 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:37.773630 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:37.773632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:37.773633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:37.773808 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:37.773877 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:37.774477 (+ 600us) server_negotiation.cc:299] Negotiation successful
0927 18:41:37.774578 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 18:41:37.776036 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:37.763578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36019 (local address 127.6.4.1:59619)
0927 18:41:37.764035 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:37.764076 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:37.765213 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:37.765685 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:37.765692 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:37.766059 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:37.766513 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:37.766526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.767277 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.767280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:37.768433 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:37.768444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.768890 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.768937 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:37.769117 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:37.769640 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:37.769655 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:37.771171 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:37.772744 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:37.772750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:37.772763 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:37.773092 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:37.773365 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:37.773368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:37.773370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:37.773551 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:37.773908 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:37.773914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:37.774230 (+ 316us) client_negotiation.cc:769] Sending connection context
0927 18:41:37.774451 (+ 221us) client_negotiation.cc:240] Negotiation successful
0927 18:41:37.774908 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":100,"threads_started":1}
I20260927 18:41:37.777271 25530 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36019
I20260927 18:41:37.777488 25530 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.4.2@KRBTEST.COM; defaulting to no policy
I20260927 18:41:37.777945 25530 heartbeater.cc:511] Master 127.6.4.62:36019 requested a full tablet report, sending...
I20260927 18:41:37.779389 25339 ts_manager.cc:194] Registered new tserver with Master: f281b98522a448b3a36a838831bd07b7 (127.6.4.1:33737)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.2@KRBTEST.COM" created.
I20260927 18:41:37.780766 25339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.1@KRBTEST.COM'} at 127.6.4.1:59619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
Entry for principal kudu/127.6.4.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab.
I20260927 18:41:37.793123 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36019
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:37.899624 25535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:37.899883 25535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:37.899925 25535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:37.903825 25535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:37.903888 25535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:37.903945 25535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:37.909046 25535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.2.keytab
--principal=kudu/127.6.4.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:37.910006 25535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:37.910929 25535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:37.919852 25543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:37.919844 25541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:37.919981 25535 server_base.cc:1034] running on GCE node
W20260927 18:41:37.919858 25540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:37.920567 25535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:37.921137 25535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:37.922297 25535 hybrid_clock.cc:648] HybridClock initialized: now 1790534497922275 us; error 32 us; skew 500 ppm
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:37.924861 25535 init.cc:377] Logged in from keytab as kudu/127.6.4.2@KRBTEST.COM (short username kudu)
I20260927 18:41:37.925659 25535 webserver.cc:460] Webserver started at http://127.6.4.2:34269/ using document root <none> and password file <none>
I20260927 18:41:37.926174 25535 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:37.926235 25535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:37.926445 25535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:37.928545 25535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "9aa9d105ebc44c398abd5d89b0edda60"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "d014bb5dc18df3ea2a89a4e46412734a"
server_key_iv: "f081bc8a440528081049adba7daa42af"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.929067 25535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "9aa9d105ebc44c398abd5d89b0edda60"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "d014bb5dc18df3ea2a89a4e46412734a"
server_key_iv: "f081bc8a440528081049adba7daa42af"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.932461 25535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260927 18:41:37.934682 25550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.935883 25535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:37.935978 25535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "9aa9d105ebc44c398abd5d89b0edda60"
format_stamp: "Formatted at 2026-09-27 18:41:37 on dist-test-slave-bzzj"
server_key: "d014bb5dc18df3ea2a89a4e46412734a"
server_key_iv: "f081bc8a440528081049adba7daa42af"
server_key_version: "encryptionkey@0"
I20260927 18:41:37.936061 25535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:37.952029 25535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:37.955044 25535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:37.955191 25535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:37.955727 25535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:37.956544 25535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:37.956588 25535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.956625 25535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:37.956642 25535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:37.964670 25535 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:41995
I20260927 18:41:37.964717 25663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:41995 every 8 connection(s)
I20260927 18:41:37.965507 25535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:37.968672 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25535
I20260927 18:41:37.968802 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:37.969117 6160 external_mini_cluster.cc:1428] Setting key fa3e9177eba7d9c000a38ece4e385960
Sep 27 18:41:37 dist-test-slave-bzzj krb5kdc[25294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534497, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.2@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.4.3@KRBTEST.COM; defaulting to no policy
I20260927 18:41:37.980377 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:37.967235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:36261 (local address 127.6.4.62:36019)
0927 18:41:37.967400 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:37.967404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:37.968399 (+ 995us) server_negotiation.cc:407] Connection header received
0927 18:41:37.970346 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:37.970350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:37.970437 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:37.970519 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:37.971823 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.972519 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.974136 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.974351 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.974427 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:37.977192 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:37.977195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:37.977198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:37.977221 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:37.978459 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:37.979000 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:37.979003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:37.979004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:37.979079 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:37.979350 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:37.979353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:37.979354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:37.979544 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:37.979619 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:37.980122 (+ 503us) server_negotiation.cc:299] Negotiation successful
0927 18:41:37.980227 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.3@KRBTEST.COM" created.
I20260927 18:41:37.981357 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:37.967665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36019 (local address 127.6.4.2:36261)
0927 18:41:37.968218 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:37.968274 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:37.969824 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:37.970827 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:37.970835 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:37.971206 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:37.971682 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:37.971695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.972649 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.972653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:37.974022 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:37.974033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:37.974440 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:37.974498 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:37.974699 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:37.975275 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:37.975295 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:37.977031 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:37.978609 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:37.978616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:37.978633 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:37.978914 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:37.979148 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:37.979150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:37.979151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:37.979278 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:37.979633 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:37.979638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:37.979904 (+ 266us) client_negotiation.cc:769] Sending connection context
0927 18:41:37.980059 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 18:41:37.980368 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":129,"threads_started":1}
I20260927 18:41:37.982452 25664 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36019
I20260927 18:41:37.982681 25664 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:37.983146 25664 heartbeater.cc:511] Master 127.6.4.62:36019 requested a full tablet report, sending...
I20260927 18:41:37.984167 25339 ts_manager.cc:194] Registered new tserver with Master: 9aa9d105ebc44c398abd5d89b0edda60 (127.6.4.2:41995)
I20260927 18:41:37.984743 25339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.2@KRBTEST.COM'} at 127.6.4.2:36261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
Entry for principal kudu/127.6.4.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab.
I20260927 18:41:37.994854 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36019
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:38.102581 25669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:38.102835 25669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:38.102876 25669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:38.106791 25669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:38.106850 25669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:38.106907 25669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:38.112057 25669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.3.keytab
--principal=kudu/127.6.4.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:38.113056 25669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:38.114017 25669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:38.123117 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:38.123308 25669 server_base.cc:1034] running on GCE node
W20260927 18:41:38.123133 25677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.123118 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:38.123814 25669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:38.124467 25669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:38.125617 25669 hybrid_clock.cc:648] HybridClock initialized: now 1790534498125599 us; error 28 us; skew 500 ppm
Sep 27 18:41:38 dist-test-slave-bzzj krb5kdc[25294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534498, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:38.128361 25669 init.cc:377] Logged in from keytab as kudu/127.6.4.3@KRBTEST.COM (short username kudu)
I20260927 18:41:38.129225 25669 webserver.cc:460] Webserver started at http://127.6.4.3:32987/ using document root <none> and password file <none>
I20260927 18:41:38.129737 25669 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:38.129828 25669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:38.130039 25669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:38.131997 25669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "945747ba247947d7ba0e84927c141f8c"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "664f084048d836ee4d902802d3011c8e"
server_key_iv: "28a31c08a2e576082d04bca42e58ecea"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.132546 25669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "945747ba247947d7ba0e84927c141f8c"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "664f084048d836ee4d902802d3011c8e"
server_key_iv: "28a31c08a2e576082d04bca42e58ecea"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.136027 25669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:38.138453 25684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.139755 25669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:38.139859 25669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "945747ba247947d7ba0e84927c141f8c"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "664f084048d836ee4d902802d3011c8e"
server_key_iv: "28a31c08a2e576082d04bca42e58ecea"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.139971 25669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:38.156215 25669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:38.159336 25669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:38.159518 25669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:38.160149 25669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:38.160996 25669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:38.161042 25669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.161094 25669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:38.161120 25669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.169461 25669 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:41451
I20260927 18:41:38.169485 25797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:41451 every 8 connection(s)
I20260927 18:41:38.170295 25669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:38.170563 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25669
I20260927 18:41:38.170670 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:38.171007 6160 external_mini_cluster.cc:1428] Setting key 4c65226a62f21cc467ba0228f92b36a4
Sep 27 18:41:38 dist-test-slave-bzzj krb5kdc[25294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534498, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.3@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:38.183100 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.172129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:51301 (local address 127.6.4.62:36019)
0927 18:41:38.172259 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.172262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.173228 (+ 966us) server_negotiation.cc:407] Connection header received
0927 18:41:38.174492 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.174495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.174550 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.174618 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:38.175805 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.176453 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.177597 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.177818 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.177902 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.180293 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:38.180296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:38.180298 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:38.180321 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:38.181454 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:38.181868 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:38.181870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:38.181871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:38.181934 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:38.182175 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:38.182177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:38.182178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:38.182337 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:38.182405 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.182881 (+ 476us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.182981 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:38.183949 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.172591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:36019 (local address 127.6.4.3:51301)
0927 18:41:38.173074 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.173132 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.174302 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.174782 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.174789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.175155 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:38.175683 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.175697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.176570 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.176573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.177490 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.177499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.177899 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.177958 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.178111 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.178638 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:41:38.178655 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:38.180080 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:38.181552 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:38.181557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:38.181568 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:38.181798 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:38.181994 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:41:38.181996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:41:38.181998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:41:38.182115 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:41:38.182403 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:38.182408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:38.182637 (+ 229us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.182786 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.183031 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":104,"threads_started":1}
I20260927 18:41:38.185039 25798 heartbeater.cc:347] Connected to a master server at 127.6.4.62:36019
I20260927 18:41:38.185247 25798 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:38.185683 25798 heartbeater.cc:511] Master 127.6.4.62:36019 requested a full tablet report, sending...
I20260927 18:41:38.186597 25338 ts_manager.cc:194] Registered new tserver with Master: 945747ba247947d7ba0e84927c141f8c (127.6.4.3:41451)
I20260927 18:41:38.187140 25338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.4.3@KRBTEST.COM'} at 127.6.4.3:51301
I20260927 18:41:38.196676 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:38.204841 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.198529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.6.4.62:36019)
0927 18:41:38.198648 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.198651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.198805 (+ 154us) server_negotiation.cc:407] Connection header received
0927 18:41:38.198911 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.198913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.198952 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.199019 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:38.199605 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.200164 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.200905 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.201106 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.201172 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.202334 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:38.202336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:38.202338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:38.202354 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:38.203468 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:38.203822 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:38.203825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:38.203826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:38.203903 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:38.204131 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:38.204134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:38.204135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:38.204344 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:38.204446 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.204609 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.204695 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:41:38.207453 25338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:38.210440 25338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:38.224076 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.219366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.6.4.1:33737)
0927 18:41:38.219764 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.219779 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.219800 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:41:38.219880 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.219884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.220063 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.220220 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:38.220432 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.221426 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.222835 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.223375 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.223470 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.223626 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.223800 (+ 174us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.223903 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":114,"threads_started":1}
I20260927 18:41:38.224079 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.218237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:41451 (local address 127.0.0.1:39168)
0927 18:41:38.218820 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.218866 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.218980 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.220279 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.220281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.220294 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:38.220357 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.220362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.221501 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.221505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.222598 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.222607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.223665 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.223669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.223765 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.223787 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.223838 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.223920 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":29,"thread_start_us":75,"threads_started":1}
I20260927 18:41:38.224071 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.218963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:33737 (local address 127.0.0.1:58204)
0927 18:41:38.219369 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.219391 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.219463 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.220225 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.220228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.220246 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:38.220331 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.220337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.221539 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.221542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.222684 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.222695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.223486 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.223489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.223590 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.223625 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.223765 (+ 140us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.223903 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260927 18:41:38.224375 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.218659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.6.4.2:41995)
0927 18:41:38.219037 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.219049 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.219073 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:41:38.219144 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.219148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.219327 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.219473 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:38.219891 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.221044 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.222796 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.223547 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.223633 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.223819 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.224129 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.224221 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":91,"threads_started":1}
I20260927 18:41:38.224570 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.218368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.6.4.3:41451)
0927 18:41:38.219890 (+ 1522us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.219900 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.219918 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:41:38.219974 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.219978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.220150 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.220479 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:38.220595 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.221376 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.222704 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.223530 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.224164 (+ 634us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.224297 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.224343 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.224432 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1414,"thread_start_us":84,"threads_started":1}
I20260927 18:41:38.224756 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.218331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41995 (local address 127.0.0.1:56372)
0927 18:41:38.218899 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.218913 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.219000 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.219578 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.219584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.219603 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:38.219736 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.219745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.221175 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.221179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.222684 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.222695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.223876 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.223880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.223955 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.223976 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.224571 (+ 595us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.224648 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":95,"thread_start_us":97,"threads_started":1}
I20260927 18:41:38.227412 25464 tablet_service.cc:1467] Processing CreateTablet for tablet 592315c38c47432593b09c682d28beea (DEFAULT_TABLE table=test-table [id=bd97047052954813aa67a971a5e58c39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:38.227432 25598 tablet_service.cc:1467] Processing CreateTablet for tablet 592315c38c47432593b09c682d28beea (DEFAULT_TABLE table=test-table [id=bd97047052954813aa67a971a5e58c39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:38.228454 25598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592315c38c47432593b09c682d28beea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:38.228456 25464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592315c38c47432593b09c682d28beea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:38.230166 25732 tablet_service.cc:1467] Processing CreateTablet for tablet 592315c38c47432593b09c682d28beea (DEFAULT_TABLE table=test-table [id=bd97047052954813aa67a971a5e58c39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:38.231179 25732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592315c38c47432593b09c682d28beea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:38.235047 25815 tablet_bootstrap.cc:492] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Bootstrap starting.
I20260927 18:41:38.236163 25816 tablet_bootstrap.cc:492] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: Bootstrap starting.
I20260927 18:41:38.237181 25815 tablet_bootstrap.cc:654] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:38.238118 25817 tablet_bootstrap.cc:492] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Bootstrap starting.
I20260927 18:41:38.238278 25816 tablet_bootstrap.cc:654] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:38.238282 25815 log.cc:824] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:38.239394 25816 log.cc:824] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:38.240461 25817 tablet_bootstrap.cc:654] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:38.240517 25815 tablet_bootstrap.cc:492] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: No bootstrap required, opened a new log
I20260927 18:41:38.240708 25815 ts_tablet_manager.cc:1397] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:38.241400 25816 tablet_bootstrap.cc:492] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: No bootstrap required, opened a new log
I20260927 18:41:38.241395 25817 log.cc:824] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:38.241575 25816 ts_tablet_manager.cc:1397] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 18:41:38.242894 25817 tablet_bootstrap.cc:492] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: No bootstrap required, opened a new log
I20260927 18:41:38.243052 25817 ts_tablet_manager.cc:1397] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 18:41:38.244638 25815 raft_consensus.cc:348] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.244935 25815 raft_consensus.cc:374] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:38.244992 25815 raft_consensus.cc:729] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa9d105ebc44c398abd5d89b0edda60, State: Initialized, Role: FOLLOWER
I20260927 18:41:38.245213 25816 raft_consensus.cc:348] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.245445 25816 raft_consensus.cc:374] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:38.245496 25816 raft_consensus.cc:729] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f281b98522a448b3a36a838831bd07b7, State: Initialized, Role: FOLLOWER
I20260927 18:41:38.245443 25815 consensus_queue.cc:260] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.245899 25816 consensus_queue.cc:260] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.245975 25817 raft_consensus.cc:348] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.246191 25817 raft_consensus.cc:374] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:38.246420 25815 ts_tablet_manager.cc:1428] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:38.246662 25530 heartbeater.cc:503] Master 127.6.4.62:36019 was elected leader, sending a full tablet report...
I20260927 18:41:38.246739 25664 heartbeater.cc:503] Master 127.6.4.62:36019 was elected leader, sending a full tablet report...
I20260927 18:41:38.246778 25816 ts_tablet_manager.cc:1428] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 18:41:38.246244 25817 raft_consensus.cc:729] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945747ba247947d7ba0e84927c141f8c, State: Initialized, Role: FOLLOWER
I20260927 18:41:38.247581 25817 consensus_queue.cc:260] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.248224 25798 heartbeater.cc:503] Master 127.6.4.62:36019 was elected leader, sending a full tablet report...
I20260927 18:41:38.248488 25817 ts_tablet_manager.cc:1428] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:41:38.256780 25823 raft_consensus.cc:482] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:38.256965 25823 raft_consensus.cc:504] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.257969 25823 leader_election.cc:288] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa9d105ebc44c398abd5d89b0edda60 (127.6.4.2:41995), f281b98522a448b3a36a838831bd07b7 (127.6.4.1:33737)
W20260927 18:41:38.262591 25531 tablet.cc:2369] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:38.262877 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.258283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:43675 (local address 127.6.4.2:41995)
0927 18:41:38.258409 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.258413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.258571 (+ 158us) server_negotiation.cc:407] Connection header received
0927 18:41:38.258695 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.258698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.258746 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.258821 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:38.259164 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.259939 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.261626 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.262246 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.262352 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.262432 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.262679 (+ 247us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.262753 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:41:38.262892 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.258147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:41995 (local address 127.6.4.3:43675)
0927 18:41:38.258493 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.258507 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.258603 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.258852 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.258854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.258881 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:38.259066 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.259073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.260043 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.260046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.261502 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.261513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.262375 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.262379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.262549 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.262582 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.262647 (+ 65us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.262733 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":240,"spinlock_wait_cycles":1280}
I20260927 18:41:38.263263 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.258294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:45471 (local address 127.6.4.1:33737)
0927 18:41:38.258407 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.258412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.258644 (+ 232us) server_negotiation.cc:407] Connection header received
0927 18:41:38.258728 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.258731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.258771 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.258836 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:38.259164 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.259950 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.261626 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.262250 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.262352 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.262420 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.263081 (+ 661us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.263156 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:41:38.263391 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.258180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:33737 (local address 127.6.4.3:45471)
0927 18:41:38.258568 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.258579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.258654 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.258845 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.258848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.258878 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:38.259071 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.259077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.260043 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.260046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.261502 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.261513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.262375 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.262379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.262524 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.262571 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.262647 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.262733 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":122,"spinlock_wait_cycles":10496,"thread_start_us":109,"threads_started":1}
I20260927 18:41:38.263563 25618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592315c38c47432593b09c682d28beea" candidate_uuid: "945747ba247947d7ba0e84927c141f8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa9d105ebc44c398abd5d89b0edda60" is_pre_election: true
I20260927 18:41:38.263883 25618 raft_consensus.cc:2454] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945747ba247947d7ba0e84927c141f8c in term 0.
I20260927 18:41:38.263913 25484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592315c38c47432593b09c682d28beea" candidate_uuid: "945747ba247947d7ba0e84927c141f8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f281b98522a448b3a36a838831bd07b7" is_pre_election: true
I20260927 18:41:38.264212 25484 raft_consensus.cc:2454] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 945747ba247947d7ba0e84927c141f8c in term 0.
I20260927 18:41:38.264312 25686 leader_election.cc:302] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 945747ba247947d7ba0e84927c141f8c, 9aa9d105ebc44c398abd5d89b0edda60; no voters:
I20260927 18:41:38.264629 25823 raft_consensus.cc:2781] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:38.264704 25823 raft_consensus.cc:482] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:38.264756 25823 raft_consensus.cc:3037] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:38.265955 25823 raft_consensus.cc:504] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.266300 25823 leader_election.cc:288] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [CANDIDATE]: Term 1 election: Requested vote from peers 9aa9d105ebc44c398abd5d89b0edda60 (127.6.4.2:41995), f281b98522a448b3a36a838831bd07b7 (127.6.4.1:33737)
I20260927 18:41:38.266620 25618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592315c38c47432593b09c682d28beea" candidate_uuid: "945747ba247947d7ba0e84927c141f8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa9d105ebc44c398abd5d89b0edda60"
I20260927 18:41:38.266681 25484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592315c38c47432593b09c682d28beea" candidate_uuid: "945747ba247947d7ba0e84927c141f8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f281b98522a448b3a36a838831bd07b7"
I20260927 18:41:38.266723 25618 raft_consensus.cc:3037] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:38.266806 25484 raft_consensus.cc:3037] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:38.267732 25618 raft_consensus.cc:2454] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945747ba247947d7ba0e84927c141f8c in term 1.
I20260927 18:41:38.267747 25484 raft_consensus.cc:2454] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 945747ba247947d7ba0e84927c141f8c in term 1.
I20260927 18:41:38.268038 25688 leader_election.cc:302] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 945747ba247947d7ba0e84927c141f8c, 9aa9d105ebc44c398abd5d89b0edda60, f281b98522a448b3a36a838831bd07b7; no voters:
I20260927 18:41:38.268239 25823 raft_consensus.cc:2781] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:38.268442 25823 raft_consensus.cc:686] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 1 LEADER]: Becoming Leader. State: Replica: 945747ba247947d7ba0e84927c141f8c, State: Running, Role: LEADER
I20260927 18:41:38.268760 25823 consensus_queue.cc:237] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } }
I20260927 18:41:38.271659 25338 catalog_manager.cc:5495] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 945747ba247947d7ba0e84927c141f8c (127.6.4.3). New cstate: current_term: 1 leader_uuid: "945747ba247947d7ba0e84927c141f8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945747ba247947d7ba0e84927c141f8c" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 41451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 } health_report { overall_health: UNKNOWN } } }
I20260927 18:41:38.285810 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.282607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39170 (local address 127.6.4.3:41451)
0927 18:41:38.282722 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.282725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.282739 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:41:38.282865 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.282868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.282922 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.283003 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:38.283183 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.283670 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.284540 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.284765 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.284845 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.285076 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:38.285490 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:38.285576 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.285620 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.285669 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:41:38.291535 25618 raft_consensus.cc:1261] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 1 FOLLOWER]: Refusing update from remote peer 945747ba247947d7ba0e84927c141f8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:38.291661 25484 raft_consensus.cc:1261] T 592315c38c47432593b09c682d28beea P f281b98522a448b3a36a838831bd07b7 [term 1 FOLLOWER]: Refusing update from remote peer 945747ba247947d7ba0e84927c141f8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:38.292292 25825 consensus_queue.cc:1035] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa9d105ebc44c398abd5d89b0edda60" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 41995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:38.292470 25823 consensus_queue.cc:1035] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f281b98522a448b3a36a838831bd07b7" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:38.309440 25337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:38.309609 25337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:38.311983 25337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f90fac210f8f487992f37f37b488b937: Sending DeleteTablet for 3 replicas of tablet 592315c38c47432593b09c682d28beea
I20260927 18:41:38.312798 25598 tablet_service.cc:1514] Processing DeleteTablet for tablet 592315c38c47432593b09c682d28beea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:38 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:56372
I20260927 18:41:38.312826 25464 tablet_service.cc:1514] Processing DeleteTablet for tablet 592315c38c47432593b09c682d28beea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:38 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:58204
I20260927 18:41:38.312847 25732 tablet_service.cc:1514] Processing DeleteTablet for tablet 592315c38c47432593b09c682d28beea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:38 UTC) from {username='kudu', principal='kudu/127.6.4.62@KRBTEST.COM'} at 127.0.0.1:39168
I20260927 18:41:38.313354 25840 tablet_replica.cc:330] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: stopping tablet replica
I20260927 18:41:38.313604 25839 tablet_replica.cc:330] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: stopping tablet replica
I20260927 18:41:38.313674 25840 raft_consensus.cc:2229] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:38.313854 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25401
I20260927 18:41:38.314188 25839 raft_consensus.cc:2229] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:38.314105 25840 raft_consensus.cc:2258] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:38.315251 25839 raft_consensus.cc:2258] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:38.315810 25840 ts_tablet_manager.cc:1905] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:38.316896 25839 ts_tablet_manager.cc:1905] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:38.318499 25840 ts_tablet_manager.cc:1918] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:38.318588 25840 log.cc:1196] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/592315c38c47432593b09c682d28beea
I20260927 18:41:38.318892 25840 ts_tablet_manager.cc:1939] T 592315c38c47432593b09c682d28beea P 945747ba247947d7ba0e84927c141f8c: Deleting consensus metadata
I20260927 18:41:38.319625 25839 ts_tablet_manager.cc:1918] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:38.319818 25839 log.cc:1196] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/592315c38c47432593b09c682d28beea
I20260927 18:41:38.320039 25326 catalog_manager.cc:4841] TS 945747ba247947d7ba0e84927c141f8c (127.6.4.3:41451): tablet 592315c38c47432593b09c682d28beea (table test-table [id=bd97047052954813aa67a971a5e58c39]) successfully deleted
I20260927 18:41:38.320253 25839 ts_tablet_manager.cc:1939] T 592315c38c47432593b09c682d28beea P 9aa9d105ebc44c398abd5d89b0edda60: Deleting consensus metadata
I20260927 18:41:38.321468 25324 catalog_manager.cc:4841] TS 9aa9d105ebc44c398abd5d89b0edda60 (127.6.4.2:41995): tablet 592315c38c47432593b09c682d28beea (table test-table [id=bd97047052954813aa67a971a5e58c39]) successfully deleted
W20260927 18:41:38.321842 25326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:33737 (error 108)
I20260927 18:41:38.322038 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25535
I20260927 18:41:38.323472 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.323183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:33737 (local address 127.0.0.1:58210)
0927 18:41:38.323309 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.323385 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:33737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260927 18:41:38.323621 25326 catalog_manager.cc:4584] TS f281b98522a448b3a36a838831bd07b7 (127.6.4.1:33737): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 592315c38c47432593b09c682d28beea: Network error: Client connection negotiation failed: client connection to 127.6.4.1:33737: connect: Connection refused (error 111)
I20260927 18:41:38.327303 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25669
I20260927 18:41:38.332198 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25307
2026-09-27T18:41:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2075 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T18:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:38Z Disabled control of system clock
I20260927 18:41:38.359730 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40375
--webserver_interface=127.6.4.62
--webserver_port=0
--builtin_ntp_servers=127.6.4.20:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:41:38.470862 25845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:38.471144 25845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:38.471196 25845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:38.475289 25845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:38.475353 25845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:38.475374 25845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:38.475394 25845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:38.475415 25845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:38.480583 25845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:38.481664 25845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:38.483068 25845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:41:38.491199 25845 server_base.cc:1034] running on GCE node
W20260927 18:41:38.491209 25851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.491210 25853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.491181 25850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:38.492064 25845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:38.493145 25845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:38.494313 25845 hybrid_clock.cc:648] HybridClock initialized: now 1790534498494295 us; error 28 us; skew 500 ppm
I20260927 18:41:38.496058 25845 webserver.cc:460] Webserver started at http://127.6.4.62:35273/ using document root <none> and password file <none>
I20260927 18:41:38.496621 25845 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:38.496687 25845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:38.496865 25845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:38.498827 25845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "5b7e3926df074aea8c56824e328695db"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "410ad6544480b9fe307d35b00669f5c2"
server_key_iv: "67da3ed73188c505c8fcf8d615d58cf9"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.499333 25845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "5b7e3926df074aea8c56824e328695db"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "410ad6544480b9fe307d35b00669f5c2"
server_key_iv: "67da3ed73188c505c8fcf8d615d58cf9"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.502724 25845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260927 18:41:38.505066 25859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.506234 25845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:41:38.506356 25845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "5b7e3926df074aea8c56824e328695db"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "410ad6544480b9fe307d35b00669f5c2"
server_key_iv: "67da3ed73188c505c8fcf8d615d58cf9"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.506450 25845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:38.537886 25845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:38.538599 25845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:38.538772 25845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:38.545447 25845 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40375
I20260927 18:41:38.545471 25911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40375 every 8 connection(s)
I20260927 18:41:38.546344 25845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:38.549319 25912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:38.554612 25912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db: Bootstrap starting.
I20260927 18:41:38.555363 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25845
I20260927 18:41:38.555490 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:38.555806 6160 external_mini_cluster.cc:1428] Setting key 6b20fc7e6eaa93d41a571f9a2c43dfe8
I20260927 18:41:38.557493 25912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:38.558198 25912 log.cc:824] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:38.560043 25912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db: No bootstrap required, opened a new log
I20260927 18:41:38.563403 25912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7e3926df074aea8c56824e328695db" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40375 } }
I20260927 18:41:38.563617 25912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:38.563678 25912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b7e3926df074aea8c56824e328695db, State: Initialized, Role: FOLLOWER
I20260927 18:41:38.564121 25912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7e3926df074aea8c56824e328695db" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40375 } }
I20260927 18:41:38.564270 25912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:38.564333 25912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:38.564425 25912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:38.564715 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.556970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.6.4.62:40375)
0927 18:41:38.557617 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.557632 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.557668 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:41:38.558303 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.558328 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.558818 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.559162 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:38.559583 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.560634 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.561504 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.561988 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.562149 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.562909 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:38.562914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:38.562926 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:38.563296 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:38.563352 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:38.563372 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:38.563454 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.563589 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.563875 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260927 18:41:38.565415 25912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7e3926df074aea8c56824e328695db" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40375 } }
I20260927 18:41:38.565749 25912 leader_election.cc:302] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b7e3926df074aea8c56824e328695db; no voters:
I20260927 18:41:38.566052 25912 leader_election.cc:288] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:38.566154 25917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:38.566318 25917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [term 1 LEADER]: Becoming Leader. State: Replica: 5b7e3926df074aea8c56824e328695db, State: Running, Role: LEADER
I20260927 18:41:38.566622 25917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7e3926df074aea8c56824e328695db" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40375 } }
I20260927 18:41:38.567284 25912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:38.567778 25917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b7e3926df074aea8c56824e328695db. Latest consensus state: current_term: 1 leader_uuid: "5b7e3926df074aea8c56824e328695db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7e3926df074aea8c56824e328695db" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40375 } } }
I20260927 18:41:38.567888 25917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:38.568137 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b7e3926df074aea8c56824e328695db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7e3926df074aea8c56824e328695db" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40375 } } }
I20260927 18:41:38.568239 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:38.568576 25921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:38.571604 25921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:38.576980 25921 catalog_manager.cc:1347] Generated new cluster ID: 78e54c41d1e94fb39a04f9b2294e8475
I20260927 18:41:38.577051 25921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:38.589398 25921 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:38.590610 25921 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:38.595110 25921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db: Generated new TSK 0
I20260927 18:41:38.595889 25921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:38.599938 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.1:0
--local_ip_for_outbound_sockets=127.6.4.1
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40375
--builtin_ntp_servers=127.6.4.20:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:41:38.707567 25936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:38.707813 25936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:38.707863 25936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:38.711860 25936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:38.711913 25936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:38.711989 25936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.1
I20260927 18:41:38.716708 25936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.4.1
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.1
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:38.717728 25936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:38.719040 25936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:41:38.728286 25936 server_base.cc:1034] running on GCE node
W20260927 18:41:38.728266 25944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.728242 25941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.728489 25942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:38.728838 25936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:38.729434 25936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:38.730593 25936 hybrid_clock.cc:648] HybridClock initialized: now 1790534498730568 us; error 36 us; skew 500 ppm
I20260927 18:41:38.732215 25936 webserver.cc:460] Webserver started at http://127.6.4.1:36211/ using document root <none> and password file <none>
I20260927 18:41:38.732748 25936 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:38.732810 25936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:38.732990 25936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:38.734948 25936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data/instance:
uuid: "90be1270e52043968f95f3504a904377"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "8a3f3bed9eb5d804f67d1281248e89db"
server_key_iv: "5c54dd53b4f126c48806eb7402e99ff0"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.735561 25936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance:
uuid: "90be1270e52043968f95f3504a904377"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "8a3f3bed9eb5d804f67d1281248e89db"
server_key_iv: "5c54dd53b4f126c48806eb7402e99ff0"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.739051 25936 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260927 18:41:38.741385 25950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.742523 25936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:38.742627 25936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal
uuid: "90be1270e52043968f95f3504a904377"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "8a3f3bed9eb5d804f67d1281248e89db"
server_key_iv: "5c54dd53b4f126c48806eb7402e99ff0"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.742717 25936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:38.755960 25936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:38.756687 25936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:38.756851 25936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:38.757390 25936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:38.758299 25936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:38.758344 25936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.758395 25936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:38.758421 25936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.766222 25936 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.1:41367
I20260927 18:41:38.766269 26063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.1:41367 every 8 connection(s)
I20260927 18:41:38.767060 25936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/data/info.pb
I20260927 18:41:38.775796 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.768821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.1:49737 (local address 127.6.4.62:40375)
0927 18:41:38.768951 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.768955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.770030 (+ 1075us) server_negotiation.cc:407] Connection header received
0927 18:41:38.771363 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.771366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.771432 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.771504 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:38.772418 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.772951 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.774059 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.774279 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.774364 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.775347 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:38.775351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:38.775353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:38.775367 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:38.775385 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:38.775393 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:38.775455 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.775586 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.775647 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:41:38.775920 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 25936
I20260927 18:41:38.776034 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-0/wal/instance
I20260927 18:41:38.776336 6160 external_mini_cluster.cc:1428] Setting key a01511c7b49ff22edc5738ab0ea4a3f1
I20260927 18:41:38.776715 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.769269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40375 (local address 127.6.4.1:49737)
0927 18:41:38.769887 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.769935 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.771161 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.771677 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.771684 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.771767 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:38.772284 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.772298 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.773065 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.773068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.773952 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.773961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.774360 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.774414 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.774565 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.775107 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:38.775109 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:38.775149 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:38.775210 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:38.775467 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:38.775475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:38.775481 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.775581 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.775875 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
I20260927 18:41:38.777815 26064 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40375
I20260927 18:41:38.778015 26064 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:38.778239 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.2:0
--local_ip_for_outbound_sockets=127.6.4.2
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40375
--builtin_ntp_servers=127.6.4.20:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:41:38.778509 26064 heartbeater.cc:511] Master 127.6.4.62:40375 requested a full tablet report, sending...
I20260927 18:41:38.779842 25876 ts_manager.cc:194] Registered new tserver with Master: 90be1270e52043968f95f3504a904377 (127.6.4.1:41367)
I20260927 18:41:38.781010 25876 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.1:49737
W20260927 18:41:38.885805 26067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:38.886052 26067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:38.886090 26067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:38.890039 26067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:38.890093 26067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:38.890148 26067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.2
I20260927 18:41:38.894799 26067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.4.2
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.2
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:38.895845 26067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:38.897666 26067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:38.908118 26073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.908160 26075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:38.908149 26072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:38.908298 26067 server_base.cc:1034] running on GCE node
I20260927 18:41:38.908730 26067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:38.909332 26067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:38.910494 26067 hybrid_clock.cc:648] HybridClock initialized: now 1790534498910468 us; error 40 us; skew 500 ppm
I20260927 18:41:38.912274 26067 webserver.cc:460] Webserver started at http://127.6.4.2:41251/ using document root <none> and password file <none>
I20260927 18:41:38.912813 26067 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:38.912879 26067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:38.913057 26067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:38.915015 26067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data/instance:
uuid: "98be7b36cdb64da0b6cf390043cd3f8e"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "fd90011ee8c0255c6c5fd0214d80ac0d"
server_key_iv: "40a0a4095fc5128bc222f52d3e1c0d8a"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.915526 26067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance:
uuid: "98be7b36cdb64da0b6cf390043cd3f8e"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "fd90011ee8c0255c6c5fd0214d80ac0d"
server_key_iv: "40a0a4095fc5128bc222f52d3e1c0d8a"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.918970 26067 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:41:38.921238 26081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.922396 26067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:38.922502 26067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal
uuid: "98be7b36cdb64da0b6cf390043cd3f8e"
format_stamp: "Formatted at 2026-09-27 18:41:38 on dist-test-slave-bzzj"
server_key: "fd90011ee8c0255c6c5fd0214d80ac0d"
server_key_iv: "40a0a4095fc5128bc222f52d3e1c0d8a"
server_key_version: "encryptionkey@0"
I20260927 18:41:38.922598 26067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:38.933825 26067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:38.934448 26067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:38.934602 26067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:38.935134 26067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:38.936020 26067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:38.936065 26067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.936132 26067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:38.936161 26067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:38.944016 26067 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.2:39821
I20260927 18:41:38.944072 26194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.2:39821 every 8 connection(s)
I20260927 18:41:38.944907 26067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/data/info.pb
I20260927 18:41:38.953581 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.946653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:55739 (local address 127.6.4.62:40375)
0927 18:41:38.946780 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:38.946784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:38.947815 (+ 1031us) server_negotiation.cc:407] Connection header received
0927 18:41:38.949181 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:38.949184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:38.949239 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:38.949311 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:38.950179 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.950700 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.951815 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.952025 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.952132 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.953126 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:38.953129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:38.953131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:38.953143 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:38.953160 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:38.953168 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:38.953225 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:38.953362 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:41:38.953431 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:41:38.954443 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:38.947039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40375 (local address 127.6.4.2:55739)
0927 18:41:38.947674 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:38.947713 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:38.948980 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:38.949484 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:38.949491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:38.949572 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:38.950044 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.950056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.950827 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.950830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.951693 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:38.951703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:38.952132 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:38.952185 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:38.952334 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:38.952869 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:38.952870 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:38.952910 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:38.952985 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:38.953234 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:38.953242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:38.953249 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:41:38.953352 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:41:38.953624 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":126,"threads_started":1}
I20260927 18:41:38.955057 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26067
I20260927 18:41:38.955189 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal/instance
I20260927 18:41:38.955489 6160 external_mini_cluster.cc:1428] Setting key d7ba2b34c2ea0f764675fa0b67aa8627
I20260927 18:41:38.955634 26195 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40375
I20260927 18:41:38.955857 26195 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:38.956499 26195 heartbeater.cc:511] Master 127.6.4.62:40375 requested a full tablet report, sending...
I20260927 18:41:38.957355 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.4.3:0
--local_ip_for_outbound_sockets=127.6.4.3
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40375
--builtin_ntp_servers=127.6.4.20:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:41:38.957477 25876 ts_manager.cc:194] Registered new tserver with Master: 98be7b36cdb64da0b6cf390043cd3f8e (127.6.4.2:39821)
I20260927 18:41:38.958015 25876 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.2:55739
W20260927 18:41:39.066838 26198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:39.067193 26198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:39.067248 26198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:39.071640 26198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:39.071712 26198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:39.071794 26198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.4.3
I20260927 18:41:39.077124 26198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.4.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.4.3
--webserver_port=0
--tserver_master_addrs=127.6.4.62:40375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.4.3
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:39.078156 26198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:39.079489 26198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:39.088603 26204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:39.088608 26203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:39.088608 26206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:39.088853 26198 server_base.cc:1034] running on GCE node
I20260927 18:41:39.089342 26198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:39.089948 26198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:39.091107 26198 hybrid_clock.cc:648] HybridClock initialized: now 1790534499091086 us; error 31 us; skew 500 ppm
I20260927 18:41:39.092932 26198 webserver.cc:460] Webserver started at http://127.6.4.3:44653/ using document root <none> and password file <none>
I20260927 18:41:39.093477 26198 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:39.093536 26198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:39.093712 26198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:39.095747 26198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data/instance:
uuid: "71d10901ed6b4e99888ca741faae2c84"
format_stamp: "Formatted at 2026-09-27 18:41:39 on dist-test-slave-bzzj"
server_key: "be2dcb1783456cf75223b1dc6ffdd19c"
server_key_iv: "5e1247ec31a43942a98772f2f1aaf297"
server_key_version: "encryptionkey@0"
I20260927 18:41:39.096297 26198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance:
uuid: "71d10901ed6b4e99888ca741faae2c84"
format_stamp: "Formatted at 2026-09-27 18:41:39 on dist-test-slave-bzzj"
server_key: "be2dcb1783456cf75223b1dc6ffdd19c"
server_key_iv: "5e1247ec31a43942a98772f2f1aaf297"
server_key_version: "encryptionkey@0"
I20260927 18:41:39.099852 26198 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:41:39.102190 26212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:39.103457 26198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:39.103564 26198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal
uuid: "71d10901ed6b4e99888ca741faae2c84"
format_stamp: "Formatted at 2026-09-27 18:41:39 on dist-test-slave-bzzj"
server_key: "be2dcb1783456cf75223b1dc6ffdd19c"
server_key_iv: "5e1247ec31a43942a98772f2f1aaf297"
server_key_version: "encryptionkey@0"
I20260927 18:41:39.103662 26198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:39.125202 26198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:39.125938 26198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:39.126101 26198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:39.126713 26198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:41:39.127650 26198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:41:39.127697 26198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:39.127750 26198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:41:39.127780 26198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:39.136358 26198 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.3:40281
I20260927 18:41:39.136379 26325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.3:40281 every 8 connection(s)
I20260927 18:41:39.137302 26198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/data/info.pb
I20260927 18:41:39.143294 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26198
I20260927 18:41:39.143404 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal/instance
I20260927 18:41:39.143664 6160 external_mini_cluster.cc:1428] Setting key 9407e13da96f46dd78099bf645d7fbb6
I20260927 18:41:39.146731 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.139103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.3:48689 (local address 127.6.4.62:40375)
0927 18:41:39.139249 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.139253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.140309 (+ 1056us) server_negotiation.cc:407] Connection header received
0927 18:41:39.141735 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.141739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.141794 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.141873 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:39.142756 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.143306 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.144623 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.144952 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.145058 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.146271 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:39.146274 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:39.146276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:39.146288 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:39.146309 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:39.146316 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:39.146372 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.146537 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.146608 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:41:39.147599 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.139498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:40375 (local address 127.6.4.3:48689)
0927 18:41:39.140168 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.140208 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:39.141494 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:39.142050 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:39.142056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:39.142142 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:41:39.142626 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.142638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.143420 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.143423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.144504 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.144516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.145081 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.145139 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.145343 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.146024 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:41:39.146025 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:41:39.146067 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:41:39.146131 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:41:39.146378 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:41:39.146382 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:41:39.146386 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:41:39.146476 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 18:41:39.146760 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260927 18:41:39.148695 26326 heartbeater.cc:347] Connected to a master server at 127.6.4.62:40375
I20260927 18:41:39.148918 26326 heartbeater.cc:464] Registering TS with master...
I20260927 18:41:39.149430 26326 heartbeater.cc:511] Master 127.6.4.62:40375 requested a full tablet report, sending...
I20260927 18:41:39.150497 25876 ts_manager.cc:194] Registered new tserver with Master: 71d10901ed6b4e99888ca741faae2c84 (127.6.4.3:40281)
I20260927 18:41:39.151093 25876 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.4.3:48689
I20260927 18:41:39.157593 6160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:41:39.163040 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.159395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49528 (local address 127.6.4.62:40375)
0927 18:41:39.159524 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.159529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.159784 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:41:39.159842 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.159844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.159885 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.159951 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:39.160196 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.160711 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.161507 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.161739 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.161824 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.162668 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:39.162671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:39.162673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:39.162685 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:39.162705 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:39.162712 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:39.162776 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.162852 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.162901 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:41:39.165645 25876 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:41:39.168865 25876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:41:39.181957 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.176917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:40281 (local address 127.0.0.1:45904)
0927 18:41:39.177383 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.177443 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:39.177570 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:39.178133 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:39.178138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:39.178157 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:39.178279 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.178284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.179434 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.179437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.180492 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.180500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.181423 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.181426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.181525 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.181567 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 18:41:39.181671 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 18:41:39.181789 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":11,"thread_start_us":87,"threads_started":1}
I20260927 18:41:39.182099 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.177058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.6.4.3:40281)
0927 18:41:39.177519 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.177530 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.177557 (+ 27us) server_negotiation.cc:407] Connection header received
0927 18:41:39.177736 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.177742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.177961 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.178106 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:39.178375 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.179324 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.180609 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.181300 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.181402 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.181589 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.181834 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.181936 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":139,"threads_started":1}
I20260927 18:41:39.182873 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.177060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.2:39821 (local address 127.0.0.1:42228)
0927 18:41:39.177897 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.177908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:39.178016 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:39.178841 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:39.178844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:39.178863 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:39.178959 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.178967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.180487 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.180491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.181323 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.181333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.182522 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.182526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.182611 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.182633 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:41:39.182690 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:41:39.182750 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":115,"threads_started":1}
I20260927 18:41:39.182955 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.178163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.6.4.2:39821)
0927 18:41:39.178465 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.178474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.178494 (+ 20us) server_negotiation.cc:407] Connection header received
0927 18:41:39.178551 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.178555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.178695 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.179105 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:39.179230 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.180350 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.181756 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.182400 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.182503 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.182673 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.182718 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.182802 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
I20260927 18:41:39.185235 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.177604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:41367 (local address 127.0.0.1:45800)
0927 18:41:39.178845 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.178857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:39.178946 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:39.179353 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:39.179356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:39.179373 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:39.179516 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.179524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.180829 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.180832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.181862 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.181872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.184869 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.184872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.184963 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.184985 (+ 22us) client_negotiation.cc:769] Sending connection context
0927 18:41:39.185042 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:41:39.185109 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"mutex_wait_us":43,"thread_start_us":82,"threads_started":1}
I20260927 18:41:39.185305 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.178481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.6.4.1:41367)
0927 18:41:39.178827 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.178838 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.178950 (+ 112us) server_negotiation.cc:407] Connection header received
0927 18:41:39.179061 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.179066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.179228 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.179371 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:39.179625 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.180729 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.181998 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.184746 (+ 2748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.184845 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.185032 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.185079 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.185164 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":128,"threads_started":1}
I20260927 18:41:39.185204 26260 tablet_service.cc:1467] Processing CreateTablet for tablet f4bdf77927b54e33aaab610591d50804 (DEFAULT_TABLE table=test-table [id=78e624ea0de74363a2b9bb8004e8c742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:39.185485 26129 tablet_service.cc:1467] Processing CreateTablet for tablet f4bdf77927b54e33aaab610591d50804 (DEFAULT_TABLE table=test-table [id=78e624ea0de74363a2b9bb8004e8c742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:39.186347 26129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bdf77927b54e33aaab610591d50804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:39.186347 26260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bdf77927b54e33aaab610591d50804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:39.187646 25998 tablet_service.cc:1467] Processing CreateTablet for tablet f4bdf77927b54e33aaab610591d50804 (DEFAULT_TABLE table=test-table [id=78e624ea0de74363a2b9bb8004e8c742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:41:39.188504 25998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bdf77927b54e33aaab610591d50804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:39.193578 26343 tablet_bootstrap.cc:492] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Bootstrap starting.
I20260927 18:41:39.194036 26344 tablet_bootstrap.cc:492] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Bootstrap starting.
I20260927 18:41:39.194852 26345 tablet_bootstrap.cc:492] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: Bootstrap starting.
I20260927 18:41:39.196935 26345 tablet_bootstrap.cc:654] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:39.196946 26343 tablet_bootstrap.cc:654] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:39.197436 26344 tablet_bootstrap.cc:654] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:39.197814 26343 log.cc:824] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:39.197814 26345 log.cc:824] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:39.198110 26344 log.cc:824] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:39.199741 26344 tablet_bootstrap.cc:492] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: No bootstrap required, opened a new log
I20260927 18:41:39.199805 26343 tablet_bootstrap.cc:492] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: No bootstrap required, opened a new log
I20260927 18:41:39.199925 26344 ts_tablet_manager.cc:1397] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260927 18:41:39.200001 26343 ts_tablet_manager.cc:1397] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260927 18:41:39.200340 26345 tablet_bootstrap.cc:492] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: No bootstrap required, opened a new log
I20260927 18:41:39.200517 26345 ts_tablet_manager.cc:1397] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260927 18:41:39.203229 26344 raft_consensus.cc:348] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.203454 26344 raft_consensus.cc:374] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:39.203538 26344 raft_consensus.cc:729] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d10901ed6b4e99888ca741faae2c84, State: Initialized, Role: FOLLOWER
I20260927 18:41:39.203977 26344 consensus_queue.cc:260] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.204048 26343 raft_consensus.cc:348] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.204048 26345 raft_consensus.cc:348] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.204311 26343 raft_consensus.cc:374] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:39.204311 26345 raft_consensus.cc:374] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:39.204358 26345 raft_consensus.cc:729] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90be1270e52043968f95f3504a904377, State: Initialized, Role: FOLLOWER
I20260927 18:41:39.204358 26343 raft_consensus.cc:729] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98be7b36cdb64da0b6cf390043cd3f8e, State: Initialized, Role: FOLLOWER
I20260927 18:41:39.204756 26345 consensus_queue.cc:260] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.204967 26326 heartbeater.cc:503] Master 127.6.4.62:40375 was elected leader, sending a full tablet report...
I20260927 18:41:39.204998 26344 ts_tablet_manager.cc:1428] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 18:41:39.205402 26064 heartbeater.cc:503] Master 127.6.4.62:40375 was elected leader, sending a full tablet report...
I20260927 18:41:39.205657 26345 ts_tablet_manager.cc:1428] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 18:41:39.204756 26343 consensus_queue.cc:260] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.206452 26195 heartbeater.cc:503] Master 127.6.4.62:40375 was elected leader, sending a full tablet report...
I20260927 18:41:39.206699 26343 ts_tablet_manager.cc:1428] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260927 18:41:39.268271 26065 tablet.cc:2369] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:41:39.388284 26327 tablet.cc:2369] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:39.428474 26351 raft_consensus.cc:482] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:41:39.428687 26351 raft_consensus.cc:504] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.429900 26351 leader_election.cc:288] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71d10901ed6b4e99888ca741faae2c84 (127.6.4.3:40281), 90be1270e52043968f95f3504a904377 (127.6.4.1:41367)
I20260927 18:41:39.433405 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.430131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:41367 (local address 127.6.4.2:35795)
0927 18:41:39.430346 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.430362 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:39.430463 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:39.430701 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:39.430703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:39.430719 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:39.430826 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.430832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.431518 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.431521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.432407 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.432416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.433008 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.433012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.433105 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.433123 (+ 18us) client_negotiation.cc:769] Sending connection context
0927 18:41:39.433184 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:41:39.433252 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":24}
I20260927 18:41:39.433400 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.430380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:35795 (local address 127.6.4.1:41367)
0927 18:41:39.430512 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.430517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.430531 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:41:39.430583 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.430585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.430622 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.430682 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:39.430932 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.431423 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.432523 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.432901 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.432991 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.433052 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.433214 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.433276 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:41:39.433444 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.430224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.4.2:36059 (local address 127.6.4.3:40281)
0927 18:41:39.430351 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.430355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.430602 (+ 247us) server_negotiation.cc:407] Connection header received
0927 18:41:39.430718 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.430721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.430759 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.430817 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:41:39.431061 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.431552 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.432567 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.432961 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.433045 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.433099 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.433269 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.433329 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 18:41:39.433467 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.430131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.3:40281 (local address 127.6.4.2:36059)
0927 18:41:39.430508 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.430520 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:41:39.430616 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:41:39.430832 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:41:39.430835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:41:39.430850 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:41:39.430961 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.430967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.431644 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.431648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.432453 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:41:39.432462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.433074 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.433077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:41:39.433164 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.433181 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:41:39.433241 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:41:39.433304 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":57,"thread_start_us":66,"threads_started":1}
I20260927 18:41:39.434015 26018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf77927b54e33aaab610591d50804" candidate_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90be1270e52043968f95f3504a904377" is_pre_election: true
I20260927 18:41:39.434021 26280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf77927b54e33aaab610591d50804" candidate_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d10901ed6b4e99888ca741faae2c84" is_pre_election: true
I20260927 18:41:39.434278 26280 raft_consensus.cc:2454] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98be7b36cdb64da0b6cf390043cd3f8e in term 0.
I20260927 18:41:39.434281 26018 raft_consensus.cc:2454] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98be7b36cdb64da0b6cf390043cd3f8e in term 0.
I20260927 18:41:39.434680 26084 leader_election.cc:302] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71d10901ed6b4e99888ca741faae2c84, 98be7b36cdb64da0b6cf390043cd3f8e; no voters:
I20260927 18:41:39.434895 26351 raft_consensus.cc:2781] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:41:39.434980 26351 raft_consensus.cc:482] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:41:39.435029 26351 raft_consensus.cc:3037] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:39.435969 26351 raft_consensus.cc:504] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.436296 26351 leader_election.cc:288] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [CANDIDATE]: Term 1 election: Requested vote from peers 71d10901ed6b4e99888ca741faae2c84 (127.6.4.3:40281), 90be1270e52043968f95f3504a904377 (127.6.4.1:41367)
I20260927 18:41:39.436610 26280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf77927b54e33aaab610591d50804" candidate_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d10901ed6b4e99888ca741faae2c84"
I20260927 18:41:39.436610 26018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf77927b54e33aaab610591d50804" candidate_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90be1270e52043968f95f3504a904377"
I20260927 18:41:39.436726 26280 raft_consensus.cc:3037] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:39.436728 26018 raft_consensus.cc:3037] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:39.437907 26280 raft_consensus.cc:2454] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98be7b36cdb64da0b6cf390043cd3f8e in term 1.
I20260927 18:41:39.437907 26018 raft_consensus.cc:2454] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98be7b36cdb64da0b6cf390043cd3f8e in term 1.
I20260927 18:41:39.438186 26082 leader_election.cc:302] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90be1270e52043968f95f3504a904377, 98be7b36cdb64da0b6cf390043cd3f8e; no voters:
I20260927 18:41:39.438349 26351 raft_consensus.cc:2781] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:39.438549 26351 raft_consensus.cc:686] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 1 LEADER]: Becoming Leader. State: Replica: 98be7b36cdb64da0b6cf390043cd3f8e, State: Running, Role: LEADER
I20260927 18:41:39.438861 26351 consensus_queue.cc:237] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } }
I20260927 18:41:39.441895 25875 catalog_manager.cc:5495] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 98be7b36cdb64da0b6cf390043cd3f8e (127.6.4.2). New cstate: current_term: 1 leader_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98be7b36cdb64da0b6cf390043cd3f8e" member_type: VOTER last_known_addr { host: "127.6.4.2" port: 39821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 } health_report { overall_health: UNKNOWN } } }
W20260927 18:41:39.445963 26196 tablet.cc:2369] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:41:39.475642 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.472566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.6.4.2:39821)
0927 18:41:39.472688 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:39.472691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:39.472704 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:41:39.472777 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:39.472779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:39.472834 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:39.472920 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:41:39.473103 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.473601 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.474415 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:39.474621 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:39.474693 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:39.474910 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:41:39.475329 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:41:39.475414 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:39.475468 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 18:41:39.475514 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:41:39.481362 26280 raft_consensus.cc:1261] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 1 FOLLOWER]: Refusing update from remote peer 98be7b36cdb64da0b6cf390043cd3f8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:39.481362 26018 raft_consensus.cc:1261] T f4bdf77927b54e33aaab610591d50804 P 90be1270e52043968f95f3504a904377 [term 1 FOLLOWER]: Refusing update from remote peer 98be7b36cdb64da0b6cf390043cd3f8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:41:39.482061 26353 consensus_queue.cc:1035] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d10901ed6b4e99888ca741faae2c84" member_type: VOTER last_known_addr { host: "127.6.4.3" port: 40281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:39.482227 26351 consensus_queue.cc:1035] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "90be1270e52043968f95f3504a904377" member_type: VOTER last_known_addr { host: "127.6.4.1" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:41:39.489898 26358 mvcc.cc:205] Tried to move back new op lower bound from 7334029309868195840 to 7334029309704413184. Current Snapshot: MvccSnapshot[applied={T|T < 7334029309868195840}]
I20260927 18:41:39.497953 25875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:39.498101 25875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:41:39.500391 25875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b7e3926df074aea8c56824e328695db: Sending DeleteTablet for 3 replicas of tablet f4bdf77927b54e33aaab610591d50804
I20260927 18:41:39.501174 26129 tablet_service.cc:1514] Processing DeleteTablet for tablet f4bdf77927b54e33aaab610591d50804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:39 UTC) from {username='slave'} at 127.0.0.1:42228
I20260927 18:41:39.501211 26260 tablet_service.cc:1514] Processing DeleteTablet for tablet f4bdf77927b54e33aaab610591d50804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:39 UTC) from {username='slave'} at 127.0.0.1:45904
I20260927 18:41:39.501662 25998 tablet_service.cc:1514] Processing DeleteTablet for tablet f4bdf77927b54e33aaab610591d50804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:41:39 UTC) from {username='slave'} at 127.0.0.1:45800
I20260927 18:41:39.501724 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25936
I20260927 18:41:39.502779 26367 tablet_replica.cc:330] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: stopping tablet replica
I20260927 18:41:39.503083 26367 raft_consensus.cc:2229] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:41:39.503398 26367 raft_consensus.cc:2258] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:39.503728 26366 tablet_replica.cc:330] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: stopping tablet replica
I20260927 18:41:39.503978 26366 raft_consensus.cc:2229] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:41:39.504379 26366 raft_consensus.cc:2258] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:41:39.505199 26367 ts_tablet_manager.cc:1905] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:39.505935 26366 ts_tablet_manager.cc:1905] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:41:39.508054 26367 ts_tablet_manager.cc:1918] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:39.508219 26367 log.cc:1196] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-2/wal/wals/f4bdf77927b54e33aaab610591d50804
I20260927 18:41:39.508527 26366 ts_tablet_manager.cc:1918] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:41:39.508615 26366 log.cc:1196] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Deleting WAL directory at /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534257539224-6160-0/minicluster-data/ts-1/wal/wals/f4bdf77927b54e33aaab610591d50804
I20260927 18:41:39.508698 26367 ts_tablet_manager.cc:1939] T f4bdf77927b54e33aaab610591d50804 P 71d10901ed6b4e99888ca741faae2c84: Deleting consensus metadata
I20260927 18:41:39.508873 26366 ts_tablet_manager.cc:1939] T f4bdf77927b54e33aaab610591d50804 P 98be7b36cdb64da0b6cf390043cd3f8e: Deleting consensus metadata
I20260927 18:41:39.509990 25862 catalog_manager.cc:4841] TS 71d10901ed6b4e99888ca741faae2c84 (127.6.4.3:40281): tablet f4bdf77927b54e33aaab610591d50804 (table test-table [id=78e624ea0de74363a2b9bb8004e8c742]) successfully deleted
I20260927 18:41:39.510226 25860 catalog_manager.cc:4841] TS 98be7b36cdb64da0b6cf390043cd3f8e (127.6.4.2:39821): tablet f4bdf77927b54e33aaab610591d50804 (table test-table [id=78e624ea0de74363a2b9bb8004e8c742]) successfully deleted
W20260927 18:41:39.510437 25860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.4.1:41367 (error 108)
I20260927 18:41:39.510560 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26067
I20260927 18:41:39.511811 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:39.511554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.1:41367 (local address 127.0.0.1:45814)
0927 18:41:39.511672 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 18:41:39.511737 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.1:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:41:39.512007 25860 catalog_manager.cc:4584] TS 90be1270e52043968f95f3504a904377 (127.6.4.1:41367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4bdf77927b54e33aaab610591d50804: Network error: Client connection negotiation failed: client connection to 127.6.4.1:41367: connect: Connection refused (error 111)
I20260927 18:41:39.515950 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26198
I20260927 18:41:39.520546 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 25845
2026-09-27T18:41:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1187 ms)
[----------] 6 tests from EncryptionPolicyTest (9960 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:39 dist-test-slave-bzzj krb5kdc[26371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:39 dist-test-slave-bzzj krb5kdc[26371](info): set up 2 sockets
Sep 27 18:41:39 dist-test-slave-bzzj krb5kdc[26371](info): commencing operation
krb5kdc: starting...
W20260927 18:41:40.459950 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:40 dist-test-slave-bzzj krb5kdc[26371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:40.572603 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:40.677562 26384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:40.677814 26384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:40.677860 26384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:40.681728 26384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:40.681784 26384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:40.681802 26384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:40.681816 26384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:40.681838 26384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:40.686832 26384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:40.687804 26384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:40.688773 26384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:40.696632 26390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:40.696650 26389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:40.696709 26392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:40.696691 26384 server_base.cc:1034] running on GCE node
I20260927 18:41:40.697391 26384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:40.698443 26384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:40.699607 26384 hybrid_clock.cc:648] HybridClock initialized: now 1790534500699587 us; error 33 us; skew 500 ppm
Sep 27 18:41:40 dist-test-slave-bzzj krb5kdc[26371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:40.702142 26384 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:40.702941 26384 webserver.cc:460] Webserver started at http://127.0.0.1:46039/ using document root <none> and password file <none>
I20260927 18:41:40.703444 26384 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:40.703504 26384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:40.703685 26384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:40.705710 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "aff75207256e4c55a0da73b8f4a7385e"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-bzzj"
server_key: "0b23d9c19dae68af940122be3a72361b"
server_key_iv: "aea2648adba8140548414e96697128f4"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.706226 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "aff75207256e4c55a0da73b8f4a7385e"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-bzzj"
server_key: "0b23d9c19dae68af940122be3a72361b"
server_key_iv: "aea2648adba8140548414e96697128f4"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.709725 26384 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260927 18:41:40.712013 26399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:40.713182 26384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:40.713290 26384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "aff75207256e4c55a0da73b8f4a7385e"
format_stamp: "Formatted at 2026-09-27 18:41:40 on dist-test-slave-bzzj"
server_key: "0b23d9c19dae68af940122be3a72361b"
server_key_iv: "aea2648adba8140548414e96697128f4"
server_key_version: "encryptionkey@0"
I20260927 18:41:40.713387 26384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:40.722889 26384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:40.725512 26384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:40.725683 26384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:40.731943 26384 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43067
I20260927 18:41:40.731979 26451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43067 every 8 connection(s)
I20260927 18:41:40.732964 26384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:40.735590 26452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:40.737944 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26384
I20260927 18:41:40.738065 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:40.738323 6160 external_mini_cluster.cc:1428] Setting key 2109f3ebb7844285be2b089410581c31
I20260927 18:41:40.741562 26452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e: Bootstrap starting.
I20260927 18:41:40.744176 26452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:40.744920 26452 log.cc:824] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:40.746701 26452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e: No bootstrap required, opened a new log
Sep 27 18:41:40 dist-test-slave-bzzj krb5kdc[26371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:41:40.749570 26452 raft_consensus.cc:348] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aff75207256e4c55a0da73b8f4a7385e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260927 18:41:40.749786 26452 raft_consensus.cc:374] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:40.749841 26452 raft_consensus.cc:729] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff75207256e4c55a0da73b8f4a7385e, State: Initialized, Role: FOLLOWER
I20260927 18:41:40.750231 26452 consensus_queue.cc:260] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aff75207256e4c55a0da73b8f4a7385e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260927 18:41:40.750355 26452 raft_consensus.cc:388] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:40.750417 26452 raft_consensus.cc:482] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:40.750510 26452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:40.751350 26452 raft_consensus.cc:504] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aff75207256e4c55a0da73b8f4a7385e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260927 18:41:40.751648 26452 leader_election.cc:302] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aff75207256e4c55a0da73b8f4a7385e; no voters:
I20260927 18:41:40.751931 26452 leader_election.cc:288] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:40.752031 26457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:40.752305 26457 raft_consensus.cc:686] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [term 1 LEADER]: Becoming Leader. State: Replica: aff75207256e4c55a0da73b8f4a7385e, State: Running, Role: LEADER
I20260927 18:41:40.752743 26457 consensus_queue.cc:237] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aff75207256e4c55a0da73b8f4a7385e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } }
I20260927 18:41:40.753266 26452 sys_catalog.cc:564] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:40.753376 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.739685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47413 (local address 127.0.0.1:43067)
0927 18:41:40.740125 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:40.740145 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:40.740188 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:41:40.741017 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:40.741040 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:40.741455 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:40.741814 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:40.742460 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.743926 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.744795 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.745432 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.745571 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.748057 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:40.748062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:40.748075 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:40.748128 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:40.750232 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.750758 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.750763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.750767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.750872 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.751180 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.751183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.751184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.751589 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:40.751778 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:40.752167 (+ 389us) server_negotiation.cc:299] Negotiation successful
0927 18:41:40.752403 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":107,"threads_started":1}
I20260927 18:41:40.754293 26458 sys_catalog.cc:455] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aff75207256e4c55a0da73b8f4a7385e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aff75207256e4c55a0da73b8f4a7385e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } } }
I20260927 18:41:40.754418 26458 sys_catalog.cc:458] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:40.754712 26459 sys_catalog.cc:455] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [sys.catalog]: SysCatalogTable state changed. Reason: New leader aff75207256e4c55a0da73b8f4a7385e. Latest consensus state: current_term: 1 leader_uuid: "aff75207256e4c55a0da73b8f4a7385e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aff75207256e4c55a0da73b8f4a7385e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43067 } } }
I20260927 18:41:40.754786 26459 sys_catalog.cc:458] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:40.756659 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260927 18:41:40.757884 26472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:40.757942 26472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:40.758013 26466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:40.758695 26466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:40.764679 26466 catalog_manager.cc:1347] Generated new cluster ID: 851d638d0b0244d798c7d86ed20cd6ff
I20260927 18:41:40.764813 26466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:40.767132 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:40.758744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43863 (local address 127.0.0.1:43067)
0927 18:41:40.758840 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:40.758844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:40.758856 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:41:40.758902 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:40.758904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:40.758945 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:40.759022 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:40.759786 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.760591 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.761660 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:40.762012 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:40.762114 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:40.763462 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:40.763466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:40.763468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:40.763493 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:40.765229 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.765731 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.765734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.765736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.765843 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:40.766283 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:40.766286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:40.766288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:40.766491 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:40.766646 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:40.766882 (+ 236us) server_negotiation.cc:299] Negotiation successful
0927 18:41:40.766980 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:40.785367 26466 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:40.786288 26466 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:40.791373 26466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aff75207256e4c55a0da73b8f4a7385e: Generated new TSK 0
I20260927 18:41:40.792032 26466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:40.798388 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26384
W20260927 18:41:40.803083 26369 connection.cc:535] client connection to 127.0.0.1:43067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43067): Connection reset by peer (error 104)
2026-09-27T18:41:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:40 dist-test-slave-bzzj krb5kdc[26483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:40 dist-test-slave-bzzj krb5kdc[26483](info): set up 2 sockets
Sep 27 18:41:40 dist-test-slave-bzzj krb5kdc[26483](info): commencing operation
krb5kdc: starting...
W20260927 18:41:41.739971 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:41 dist-test-slave-bzzj krb5kdc[26483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:41.851536 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:41.957664 26496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:41.957937 26496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:41.957998 26496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:41.961992 26496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:41.962055 26496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:41.962082 26496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:41.962103 26496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:41.962128 26496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:41.967348 26496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:41.968461 26496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:41.969431 26496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:41.977571 26502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.977767 26496 server_base.cc:1034] running on GCE node
W20260927 18:41:41.977576 26501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:41.977617 26504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:41.978446 26496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:41.979514 26496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:41.980671 26496 hybrid_clock.cc:648] HybridClock initialized: now 1790534501980655 us; error 26 us; skew 500 ppm
Sep 27 18:41:41 dist-test-slave-bzzj krb5kdc[26483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:41.996124 26496 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:41.997160 26496 webserver.cc:460] Webserver started at http://127.0.0.1:46309/ using document root <none> and password file <none>
I20260927 18:41:41.997742 26496 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:41.997813 26496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:41.998030 26496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:42.000167 26496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "e7af4ac5c257408db4fd10e09bfe0e82"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-bzzj"
server_key: "e67035fad0d4d1d0a0e313e0758dc53c"
server_key_iv: "a65abb5e6131a6d867e6def004e5b601"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.000721 26496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "e7af4ac5c257408db4fd10e09bfe0e82"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-bzzj"
server_key: "e67035fad0d4d1d0a0e313e0758dc53c"
server_key_iv: "a65abb5e6131a6d867e6def004e5b601"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.004405 26496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:41:42.006824 26511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:42.008066 26496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:42.008230 26496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "e7af4ac5c257408db4fd10e09bfe0e82"
format_stamp: "Formatted at 2026-09-27 18:41:41 on dist-test-slave-bzzj"
server_key: "e67035fad0d4d1d0a0e313e0758dc53c"
server_key_iv: "a65abb5e6131a6d867e6def004e5b601"
server_key_version: "encryptionkey@0"
I20260927 18:41:42.008332 26496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:42.029592 26496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:42.032750 26496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:42.032935 26496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:42.039633 26496 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41821
I20260927 18:41:42.039652 26563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41821 every 8 connection(s)
I20260927 18:41:42.040575 26496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:42.043138 26564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:42.047883 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26496
I20260927 18:41:42.048043 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:42.048380 6160 external_mini_cluster.cc:1428] Setting key cc5a1fd0fafefbfa8ac939ca5fa7ef16
I20260927 18:41:42.048548 26564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82: Bootstrap starting.
I20260927 18:41:42.050909 26564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:42.051600 26564 log.cc:824] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:42.053392 26564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82: No bootstrap required, opened a new log
I20260927 18:41:42.056867 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.049681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40661 (local address 127.0.0.1:41821)
0927 18:41:42.050114 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:42.050129 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:42.050163 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:41:42.050778 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:42.050794 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:42.051146 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:42.051454 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:42.051840 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.052959 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.053837 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.054290 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.054440 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.055224 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:42.055229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:42.055242 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:42.055617 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:42.055669 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:42.055689 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:42.055796 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:42.055992 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 18:41:42.056155 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":141,"threads_started":1}
I20260927 18:41:42.056815 26564 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260927 18:41:42.057054 26564 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:42.057111 26564 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7af4ac5c257408db4fd10e09bfe0e82, State: Initialized, Role: FOLLOWER
I20260927 18:41:42.057550 26564 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260927 18:41:42.057683 26564 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:42.057732 26564 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:42.057813 26564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:42.058717 26564 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260927 18:41:42.059041 26564 leader_election.cc:302] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7af4ac5c257408db4fd10e09bfe0e82; no voters:
I20260927 18:41:42.059355 26564 leader_election.cc:288] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:42.059427 26569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:42.059626 26569 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [term 1 LEADER]: Becoming Leader. State: Replica: e7af4ac5c257408db4fd10e09bfe0e82, State: Running, Role: LEADER
I20260927 18:41:42.059654 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:42.059906 26569 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260927 18:41:42.060660 26564 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:42.061089 26569 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7af4ac5c257408db4fd10e09bfe0e82. Latest consensus state: current_term: 1 leader_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } } }
I20260927 18:41:42.061193 26569 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:42.061453 26570 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7af4ac5c257408db4fd10e09bfe0e82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } } }
I20260927 18:41:42.061545 26570 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:42.061717 26577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:42.064787 26577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:42.066349 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:42.062236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48099 (local address 127.0.0.1:41821)
0927 18:41:42.062330 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:42.062335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:42.062516 (+ 181us) server_negotiation.cc:407] Connection header received
0927 18:41:42.062638 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:42.062641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:42.062685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:42.062754 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:42.063015 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.063757 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.064581 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:42.064889 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:42.064988 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:42.065954 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:42.065958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:42.065961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:42.065973 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:42.066000 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:42.066013 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:42.066099 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:42.066176 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:41:42.066208 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:41:42.069732 26577 catalog_manager.cc:1347] Generated new cluster ID: 002984a0aa3b4e9bb1205fe80de72dea
I20260927 18:41:42.069782 26577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:42.080209 26577 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:42.081056 26577 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:42.084465 26577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7af4ac5c257408db4fd10e09bfe0e82: Generated new TSK 0
I20260927 18:41:42.085193 26577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:42.094086 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26496
W20260927 18:41:42.098794 26481 connection.cc:535] client connection to 127.0.0.1:41821 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41821): Connection reset by peer (error 104)
2026-09-27T18:41:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:42 dist-test-slave-bzzj krb5kdc[26595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:42 dist-test-slave-bzzj krb5kdc[26595](info): set up 2 sockets
Sep 27 18:41:42 dist-test-slave-bzzj krb5kdc[26595](info): commencing operation
krb5kdc: starting...
W20260927 18:41:43.047881 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:43 dist-test-slave-bzzj krb5kdc[26595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:43.161252 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:43.267669 26608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:43.267938 26608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:43.267992 26608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:43.271895 26608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:43.271960 26608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:43.271984 26608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:43.272004 26608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:43.272025 26608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:43.277311 26608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:43.278327 26608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:43.279305 26608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:43.287328 26616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:43.287328 26613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:43.287345 26614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:43.287701 26608 server_base.cc:1034] running on GCE node
I20260927 18:41:43.288283 26608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:43.289314 26608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:43.290469 26608 hybrid_clock.cc:648] HybridClock initialized: now 1790534503290452 us; error 28 us; skew 500 ppm
Sep 27 18:41:43 dist-test-slave-bzzj krb5kdc[26595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:43.293081 26608 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:43.294028 26608 webserver.cc:460] Webserver started at http://127.0.0.1:35197/ using document root <none> and password file <none>
I20260927 18:41:43.294550 26608 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:43.294616 26608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:43.294807 26608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:43.296833 26608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "96b41efc3c9f49699c66cbde9aa4490b"
format_stamp: "Formatted at 2026-09-27 18:41:43 on dist-test-slave-bzzj"
server_key: "8dd36ac5eaf3b6429faf8061284a5aca"
server_key_iv: "765a0ea89e6648da9c47285936f5f1de"
server_key_version: "encryptionkey@0"
I20260927 18:41:43.297343 26608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "96b41efc3c9f49699c66cbde9aa4490b"
format_stamp: "Formatted at 2026-09-27 18:41:43 on dist-test-slave-bzzj"
server_key: "8dd36ac5eaf3b6429faf8061284a5aca"
server_key_iv: "765a0ea89e6648da9c47285936f5f1de"
server_key_version: "encryptionkey@0"
I20260927 18:41:43.300876 26608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:43.303146 26623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:43.304379 26608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:43.304498 26608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "96b41efc3c9f49699c66cbde9aa4490b"
format_stamp: "Formatted at 2026-09-27 18:41:43 on dist-test-slave-bzzj"
server_key: "8dd36ac5eaf3b6429faf8061284a5aca"
server_key_iv: "765a0ea89e6648da9c47285936f5f1de"
server_key_version: "encryptionkey@0"
I20260927 18:41:43.304601 26608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:43.308967 26608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:43.311637 26608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:43.311796 26608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:43.318218 26608 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33065
I20260927 18:41:43.318233 26675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33065 every 8 connection(s)
I20260927 18:41:43.319096 26608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:43.321731 26676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:43.326876 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26608
I20260927 18:41:43.327009 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:43.327113 26676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b: Bootstrap starting.
I20260927 18:41:43.327270 6160 external_mini_cluster.cc:1428] Setting key a7f940efc0d99c68b585aa4b026070e0
I20260927 18:41:43.329507 26676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:43.330212 26676 log.cc:824] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:43.332068 26676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b: No bootstrap required, opened a new log
I20260927 18:41:43.333433 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.328583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45575 (local address 127.0.0.1:33065)
0927 18:41:43.328980 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:43.328996 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:43.329031 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:41:43.329649 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:43.329671 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:43.329961 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:43.330225 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:43.332249 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:43.332254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:43.332277 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:43.332288 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:43.332346 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:43.332366 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:43.332457 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:43.332594 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:41:43.332736 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260927 18:41:43.335799 26676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b41efc3c9f49699c66cbde9aa4490b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260927 18:41:43.336077 26676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:43.336194 26676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b41efc3c9f49699c66cbde9aa4490b, State: Initialized, Role: FOLLOWER
I20260927 18:41:43.336302 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:43.336612 26676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b41efc3c9f49699c66cbde9aa4490b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260927 18:41:43.336728 26676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:43.336787 26676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:43.336875 26676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:43.337793 26676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b41efc3c9f49699c66cbde9aa4490b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260927 18:41:43.338106 26676 leader_election.cc:302] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b41efc3c9f49699c66cbde9aa4490b; no voters:
I20260927 18:41:43.338505 26676 leader_election.cc:288] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:43.338563 26686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:43.338774 26686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [term 1 LEADER]: Becoming Leader. State: Replica: 96b41efc3c9f49699c66cbde9aa4490b, State: Running, Role: LEADER
I20260927 18:41:43.339111 26686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b41efc3c9f49699c66cbde9aa4490b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260927 18:41:43.339833 26676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:43.340230 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:43.338268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57569 (local address 127.0.0.1:33065)
0927 18:41:43.338381 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:43.338385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:43.338651 (+ 266us) server_negotiation.cc:407] Connection header received
0927 18:41:43.338840 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:43.338843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:43.338877 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:43.338923 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:43.339804 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:43.339808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:43.339810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:43.339821 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:43.339876 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:43.339890 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:43.339971 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:43.340061 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 18:41:43.340119 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:41:43.340272 26686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96b41efc3c9f49699c66cbde9aa4490b. Latest consensus state: current_term: 1 leader_uuid: "96b41efc3c9f49699c66cbde9aa4490b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b41efc3c9f49699c66cbde9aa4490b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } } }
I20260927 18:41:43.340379 26686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:43.340844 26687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96b41efc3c9f49699c66cbde9aa4490b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b41efc3c9f49699c66cbde9aa4490b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } } }
I20260927 18:41:43.340993 26687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:43.341256 26695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:43.344305 26695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:43.349555 26695 catalog_manager.cc:1347] Generated new cluster ID: 2959cb9e7b2e461c9ebf2bd59aa43a8f
I20260927 18:41:43.349620 26695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:43.385865 26695 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:43.386184 26695 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:43.390556 26695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96b41efc3c9f49699c66cbde9aa4490b: Generated new TSK 0
I20260927 18:41:43.391327 26695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:43.397255 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26608
2026-09-27T18:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:43 dist-test-slave-bzzj krb5kdc[26707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:43 dist-test-slave-bzzj krb5kdc[26707](info): set up 2 sockets
Sep 27 18:41:43 dist-test-slave-bzzj krb5kdc[26707](info): commencing operation
krb5kdc: starting...
W20260927 18:41:44.347872 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:44 dist-test-slave-bzzj krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:44.462849 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42805
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:44.569423 26720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:44.569690 26720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:44.569748 26720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:44.573693 26720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:44.573784 26720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:44.573813 26720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:44.573850 26720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:44.573880 26720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:44.579126 26720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:44.580214 26720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:44.581219 26720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:44.589566 26726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:44.589551 26728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:44.589710 26720 server_base.cc:1034] running on GCE node
W20260927 18:41:44.589574 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:44.590373 26720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:44.591444 26720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:44.592602 26720 hybrid_clock.cc:648] HybridClock initialized: now 1790534504592586 us; error 27 us; skew 500 ppm
Sep 27 18:41:44 dist-test-slave-bzzj krb5kdc[26707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534504, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:44.595110 26720 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:44.595937 26720 webserver.cc:460] Webserver started at http://127.0.0.1:46239/ using document root <none> and password file <none>
I20260927 18:41:44.596501 26720 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:44.596566 26720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:44.596748 26720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:44.598719 26720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "457b360e7e5040f58d3a8baf21ade993"
format_stamp: "Formatted at 2026-09-27 18:41:44 on dist-test-slave-bzzj"
server_key: "9dd9b7db2d2a434334193b44c7d681ba"
server_key_iv: "1ec48a96861c04f63040650ec98cceb0"
server_key_version: "encryptionkey@0"
I20260927 18:41:44.599237 26720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "457b360e7e5040f58d3a8baf21ade993"
format_stamp: "Formatted at 2026-09-27 18:41:44 on dist-test-slave-bzzj"
server_key: "9dd9b7db2d2a434334193b44c7d681ba"
server_key_iv: "1ec48a96861c04f63040650ec98cceb0"
server_key_version: "encryptionkey@0"
I20260927 18:41:44.602725 26720 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260927 18:41:44.605073 26735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:44.606209 26720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:44.606317 26720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "457b360e7e5040f58d3a8baf21ade993"
format_stamp: "Formatted at 2026-09-27 18:41:44 on dist-test-slave-bzzj"
server_key: "9dd9b7db2d2a434334193b44c7d681ba"
server_key_iv: "1ec48a96861c04f63040650ec98cceb0"
server_key_version: "encryptionkey@0"
I20260927 18:41:44.606413 26720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:44.626652 26720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:44.629666 26720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:44.629840 26720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:44.636139 26720 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42805
I20260927 18:41:44.636175 26787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42805 every 8 connection(s)
I20260927 18:41:44.637048 26720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:44.638453 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26720
I20260927 18:41:44.638566 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:44.638827 6160 external_mini_cluster.cc:1428] Setting key b7f39df1070069691e33116eedfcab90
I20260927 18:41:44.639972 26788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:44.646520 26788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993: Bootstrap starting.
Sep 27 18:41:44 dist-test-slave-bzzj krb5kdc[26707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:41:44.649065 26788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:44.649920 26788 log.cc:824] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:44.652132 26788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993: No bootstrap required, opened a new log
I20260927 18:41:44.653106 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:44.640050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58499 (local address 127.0.0.1:42805)
0927 18:41:44.640484 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:44.640503 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:44.640545 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:41:44.641377 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:44.641399 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:44.641828 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:44.642179 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:44.642806 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:44.644275 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:44.645076 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:44.645776 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:44.645927 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:44.648425 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:44.648431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:44.648455 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:44.648485 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:44.650239 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:44.650690 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:44.650695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:44.650701 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:44.650831 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:44.651058 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:44.651061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:44.651062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:44.651547 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:44.651736 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:44.651898 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 18:41:44.652204 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":114,"threads_started":1}
I20260927 18:41:44.655845 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:44.656207 26788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457b360e7e5040f58d3a8baf21ade993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42805 } }
I20260927 18:41:44.656421 26788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:44.656477 26788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457b360e7e5040f58d3a8baf21ade993, State: Initialized, Role: FOLLOWER
I20260927 18:41:44.656857 26788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457b360e7e5040f58d3a8baf21ade993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42805 } }
I20260927 18:41:44.656987 26788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:44.657050 26788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:44.657143 26788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:44.658146 26788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457b360e7e5040f58d3a8baf21ade993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42805 } }
I20260927 18:41:44.658506 26788 leader_election.cc:302] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 457b360e7e5040f58d3a8baf21ade993; no voters:
I20260927 18:41:44.658849 26788 leader_election.cc:288] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:44.658926 26798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:44.659139 26798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [term 1 LEADER]: Becoming Leader. State: Replica: 457b360e7e5040f58d3a8baf21ade993, State: Running, Role: LEADER
I20260927 18:41:44.659478 26798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457b360e7e5040f58d3a8baf21ade993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42805 } }
I20260927 18:41:44.660162 26788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:44.660931 26799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "457b360e7e5040f58d3a8baf21ade993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457b360e7e5040f58d3a8baf21ade993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42805 } } }
I20260927 18:41:44.661046 26799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:44.661242 26799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457b360e7e5040f58d3a8baf21ade993. Latest consensus state: current_term: 1 leader_uuid: "457b360e7e5040f58d3a8baf21ade993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "457b360e7e5040f58d3a8baf21ade993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42805 } } }
I20260927 18:41:44.661309 26799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993 [sys.catalog]: This master's current role is: LEADER
W20260927 18:41:44.665290 26814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:44.665419 26814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:44.665493 26807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:44.666077 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:44.657854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60595 (local address 127.0.0.1:42805)
0927 18:41:44.657959 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:44.657963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:44.658186 (+ 223us) server_negotiation.cc:407] Connection header received
0927 18:41:44.658311 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:44.658314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:44.658354 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:44.658416 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:44.659190 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:44.659926 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:44.661076 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:44.661396 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:44.661481 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:44.663045 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:44.663048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:44.663050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:44.663069 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:44.664432 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:44.664800 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:44.664803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:44.664805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:44.664876 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:44.665109 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:44.665112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:44.665114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:44.665280 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:44.665396 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:44.665875 (+ 479us) server_negotiation.cc:299] Negotiation successful
0927 18:41:44.665972 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:41:44.666134 26807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:44.671622 26807 catalog_manager.cc:1347] Generated new cluster ID: 449a6a917ce44494b3ebe9db06b91d43
I20260927 18:41:44.671679 26807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:44.688431 26807 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:44.689594 26807 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:44.696422 26807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 457b360e7e5040f58d3a8baf21ade993: Generated new TSK 0
I20260927 18:41:44.697326 26807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:44.711706 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26720
2026-09-27T18:41:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:44 dist-test-slave-bzzj krb5kdc[26819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:44 dist-test-slave-bzzj krb5kdc[26819](info): set up 2 sockets
Sep 27 18:41:44 dist-test-slave-bzzj krb5kdc[26819](info): commencing operation
krb5kdc: starting...
W20260927 18:41:45.655933 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:45 dist-test-slave-bzzj krb5kdc[26819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:45.768554 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:45.873392 26832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:45.873652 26832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:45.873693 26832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:45.877605 26832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:45.877662 26832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:45.877679 26832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:45.877694 26832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:45.877707 26832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:45.882827 26832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:45.883841 26832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:45.884805 26832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:45.892964 26840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:45.892964 26838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:45.893002 26837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:45.893115 26832 server_base.cc:1034] running on GCE node
I20260927 18:41:45.893778 26832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:45.894850 26832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:45.896015 26832 hybrid_clock.cc:648] HybridClock initialized: now 1790534505895997 us; error 28 us; skew 500 ppm
Sep 27 18:41:45 dist-test-slave-bzzj krb5kdc[26819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:45.898586 26832 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:45.899415 26832 webserver.cc:460] Webserver started at http://127.0.0.1:44139/ using document root <none> and password file <none>
I20260927 18:41:45.899916 26832 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:45.899968 26832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:45.900167 26832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:45.902081 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "1840bd0f0e104ee5bc1f0052494be63e"
format_stamp: "Formatted at 2026-09-27 18:41:45 on dist-test-slave-bzzj"
server_key: "9c79211dbccce52f2bead1602ac13fd0"
server_key_iv: "4e9a5fd4f8c90731acb0d475a6812d98"
server_key_version: "encryptionkey@0"
I20260927 18:41:45.902550 26832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "1840bd0f0e104ee5bc1f0052494be63e"
format_stamp: "Formatted at 2026-09-27 18:41:45 on dist-test-slave-bzzj"
server_key: "9c79211dbccce52f2bead1602ac13fd0"
server_key_iv: "4e9a5fd4f8c90731acb0d475a6812d98"
server_key_version: "encryptionkey@0"
I20260927 18:41:45.906122 26832 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:45.908365 26847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:45.909482 26832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:45.909574 26832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "1840bd0f0e104ee5bc1f0052494be63e"
format_stamp: "Formatted at 2026-09-27 18:41:45 on dist-test-slave-bzzj"
server_key: "9c79211dbccce52f2bead1602ac13fd0"
server_key_iv: "4e9a5fd4f8c90731acb0d475a6812d98"
server_key_version: "encryptionkey@0"
I20260927 18:41:45.909657 26832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:45.933631 26832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:45.936381 26832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:45.936547 26832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:45.942935 26832 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37759
I20260927 18:41:45.942955 26899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37759 every 8 connection(s)
I20260927 18:41:45.943825 26832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:45.944120 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26832
I20260927 18:41:45.944252 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:45.944566 6160 external_mini_cluster.cc:1428] Setting key b6530b3796e6cf0501c0fb4a00eb15fa
I20260927 18:41:45.946808 26900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:45.952744 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e: Bootstrap starting.
I20260927 18:41:45.952970 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:45.945854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51169 (local address 127.0.0.1:37759)
0927 18:41:45.946231 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:45.946246 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:45.946282 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:41:45.946928 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:45.946945 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:45.947277 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:45.947556 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:45.947865 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:45.948864 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:45.950003 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:45.950451 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:45.950557 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:45.951350 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:45.951354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:45.951366 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:45.951723 (+ 357us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:45.951782 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:45.951804 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:45.951872 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:45.952018 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 18:41:45.952231 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260927 18:41:45.954867 26900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:45.955474 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:45.955586 26900 log.cc:824] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:45.957727 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e: No bootstrap required, opened a new log
I20260927 18:41:45.961328 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:45.957545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44633 (local address 127.0.0.1:37759)
0927 18:41:45.957646 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:45.957649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:45.957807 (+ 158us) server_negotiation.cc:407] Connection header received
0927 18:41:45.957918 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:45.957920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:45.957960 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:45.958025 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:45.958328 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:45.958817 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:45.959891 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:45.960143 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:45.960224 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:45.960977 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:45.960980 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:45.960982 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:45.960993 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:45.961014 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:45.961024 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:45.961072 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:45.961169 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 18:41:45.961212 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:41:45.961934 26900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1840bd0f0e104ee5bc1f0052494be63e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260927 18:41:45.962152 26900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:45.962206 26900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1840bd0f0e104ee5bc1f0052494be63e, State: Initialized, Role: FOLLOWER
I20260927 18:41:45.962589 26900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1840bd0f0e104ee5bc1f0052494be63e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260927 18:41:45.962706 26900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:45.962767 26900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:45.962862 26900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:45.963685 26900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1840bd0f0e104ee5bc1f0052494be63e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260927 18:41:45.964006 26900 leader_election.cc:302] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1840bd0f0e104ee5bc1f0052494be63e; no voters:
I20260927 18:41:45.964288 26900 leader_election.cc:288] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:45.964381 26910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:45.964603 26910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [term 1 LEADER]: Becoming Leader. State: Replica: 1840bd0f0e104ee5bc1f0052494be63e, State: Running, Role: LEADER
I20260927 18:41:45.964931 26910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1840bd0f0e104ee5bc1f0052494be63e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260927 18:41:45.965502 26900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:45.966496 26912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1840bd0f0e104ee5bc1f0052494be63e. Latest consensus state: current_term: 1 leader_uuid: "1840bd0f0e104ee5bc1f0052494be63e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1840bd0f0e104ee5bc1f0052494be63e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } } }
I20260927 18:41:45.966622 26912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:45.966869 26911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1840bd0f0e104ee5bc1f0052494be63e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1840bd0f0e104ee5bc1f0052494be63e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } } }
I20260927 18:41:45.966960 26911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:45.967057 26919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:45.969695 26919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:45.975418 26919 catalog_manager.cc:1347] Generated new cluster ID: 25ab8087ee574fdca855b2f2033409ce
I20260927 18:41:45.975492 26919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:45.985574 26919 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:45.986815 26919 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:45.997285 26919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1840bd0f0e104ee5bc1f0052494be63e: Generated new TSK 0
I20260927 18:41:45.998119 26919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:46.003253 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26832
2026-09-27T18:41:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:46 dist-test-slave-bzzj krb5kdc[26931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:46 dist-test-slave-bzzj krb5kdc[26931](info): set up 2 sockets
Sep 27 18:41:46 dist-test-slave-bzzj krb5kdc[26931](info): commencing operation
krb5kdc: starting...
W20260927 18:41:46.955973 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:47 dist-test-slave-bzzj krb5kdc[26931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:47.071089 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:47.176997 26944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:47.177255 26944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:47.177295 26944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:47.181280 26944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:47.181340 26944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:47.181358 26944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:47.181373 26944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:47.181388 26944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:47.186726 26944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:47.187785 26944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:47.188796 26944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:47.196926 26950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:47.196969 26949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:47.196934 26952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:47.197127 26944 server_base.cc:1034] running on GCE node
I20260927 18:41:47.197827 26944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:47.198921 26944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:47.200134 26944 hybrid_clock.cc:648] HybridClock initialized: now 1790534507200116 us; error 32 us; skew 500 ppm
Sep 27 18:41:47 dist-test-slave-bzzj krb5kdc[26931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:47.208797 26944 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:47.209817 26944 webserver.cc:460] Webserver started at http://127.0.0.1:33011/ using document root <none> and password file <none>
I20260927 18:41:47.210393 26944 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:47.210459 26944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:47.210651 26944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:47.212735 26944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "6d4a2b0f449a41abbd4c9638d6e82f70"
format_stamp: "Formatted at 2026-09-27 18:41:47 on dist-test-slave-bzzj"
server_key: "1b89b6973cbf6ff83d2c611897409a8a"
server_key_iv: "9cd49c82ad749742238520bddccbc8e3"
server_key_version: "encryptionkey@0"
I20260927 18:41:47.213270 26944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "6d4a2b0f449a41abbd4c9638d6e82f70"
format_stamp: "Formatted at 2026-09-27 18:41:47 on dist-test-slave-bzzj"
server_key: "1b89b6973cbf6ff83d2c611897409a8a"
server_key_iv: "9cd49c82ad749742238520bddccbc8e3"
server_key_version: "encryptionkey@0"
I20260927 18:41:47.216985 26944 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:47.219336 26959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:47.220643 26944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260927 18:41:47.220755 26944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "6d4a2b0f449a41abbd4c9638d6e82f70"
format_stamp: "Formatted at 2026-09-27 18:41:47 on dist-test-slave-bzzj"
server_key: "1b89b6973cbf6ff83d2c611897409a8a"
server_key_iv: "9cd49c82ad749742238520bddccbc8e3"
server_key_version: "encryptionkey@0"
I20260927 18:41:47.220851 26944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:47.225304 26944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:47.228284 26944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:47.228452 26944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:47.235116 26944 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46781
I20260927 18:41:47.235131 27011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46781 every 8 connection(s)
I20260927 18:41:47.236020 26944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:47.236636 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 26944
I20260927 18:41:47.236759 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:47.237022 6160 external_mini_cluster.cc:1428] Setting key 31a39cbd169545d217064b32bd6ab0a0
I20260927 18:41:47.239053 27012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:47.241984 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:47.238343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45779 (local address 127.0.0.1:46781)
0927 18:41:47.238830 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:47.238845 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:47.238882 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:41:47.239524 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:47.239541 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:47.239824 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:47.240144 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:47.240878 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:47.240882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:47.240893 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:47.240903 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:47.240953 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:47.240972 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:47.241091 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:47.241157 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:41:47.241302 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":141,"threads_started":1}
I20260927 18:41:47.244674 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:47.244899 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70: Bootstrap starting.
I20260927 18:41:47.247135 27012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:47.247905 27012 log.cc:824] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:47.248167 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:47.246596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59221 (local address 127.0.0.1:46781)
0927 18:41:47.246710 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:47.246714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:47.246935 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:41:47.247049 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:47.247052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:47.247085 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:47.247149 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:47.247784 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:47.247788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:47.247790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:47.247801 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:47.247827 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:47.247837 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:47.247906 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:47.247993 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 18:41:47.248029 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:41:47.249884 27012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70: No bootstrap required, opened a new log
I20260927 18:41:47.253283 27012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46781 } }
I20260927 18:41:47.253495 27012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:47.253546 27012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4a2b0f449a41abbd4c9638d6e82f70, State: Initialized, Role: FOLLOWER
I20260927 18:41:47.253948 27012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46781 } }
I20260927 18:41:47.254065 27012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:47.254123 27012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:47.254211 27012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:47.255252 27012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46781 } }
I20260927 18:41:47.255548 27012 leader_election.cc:302] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d4a2b0f449a41abbd4c9638d6e82f70; no voters:
I20260927 18:41:47.255810 27012 leader_election.cc:288] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:47.255913 27022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:47.256148 27022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [term 1 LEADER]: Becoming Leader. State: Replica: 6d4a2b0f449a41abbd4c9638d6e82f70, State: Running, Role: LEADER
I20260927 18:41:47.256428 27022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46781 } }
I20260927 18:41:47.256955 27012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:47.257881 27023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46781 } } }
I20260927 18:41:47.257997 27023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:47.258252 27024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d4a2b0f449a41abbd4c9638d6e82f70. Latest consensus state: current_term: 1 leader_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4a2b0f449a41abbd4c9638d6e82f70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46781 } } }
I20260927 18:41:47.258347 27024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:47.258440 27031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:47.261480 27031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:47.266867 27031 catalog_manager.cc:1347] Generated new cluster ID: 586d098901e1478682405736aae729fb
I20260927 18:41:47.266942 27031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:47.286912 27031 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:47.287210 27031 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:47.295083 27031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d4a2b0f449a41abbd4c9638d6e82f70: Generated new TSK 0
I20260927 18:41:47.295734 27031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:47.303146 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 26944
2026-09-27T18:41:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:47 dist-test-slave-bzzj krb5kdc[27043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:47 dist-test-slave-bzzj krb5kdc[27043](info): set up 2 sockets
Sep 27 18:41:47 dist-test-slave-bzzj krb5kdc[27043](info): commencing operation
krb5kdc: starting...
W20260927 18:41:48.247970 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:48 dist-test-slave-bzzj krb5kdc[27043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:48.364432 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:48.473722 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:48.474017 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:48.474058 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:48.478092 27056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:48.478157 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:48.478178 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:48.478207 27056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:48.478235 27056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:48.485059 27056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:48.486644 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:48.488307 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:48.498497 27061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:48.498539 27062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:48.498734 27056 server_base.cc:1034] running on GCE node
W20260927 18:41:48.498545 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:48.499605 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:48.500720 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:48.501869 27056 hybrid_clock.cc:648] HybridClock initialized: now 1790534508501846 us; error 44 us; skew 500 ppm
Sep 27 18:41:48 dist-test-slave-bzzj krb5kdc[27043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:48.504493 27056 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:48.505335 27056 webserver.cc:460] Webserver started at http://127.0.0.1:32843/ using document root <none> and password file <none>
I20260927 18:41:48.505856 27056 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:48.505916 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:48.506107 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:48.508065 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "64a3ed32bd5249d48932c40b1200f7f9"
format_stamp: "Formatted at 2026-09-27 18:41:48 on dist-test-slave-bzzj"
server_key: "c932abb3dd9df525d7b67eb0b815ea4d"
server_key_iv: "77065d30a739308ec872a50f2130567d"
server_key_version: "encryptionkey@0"
I20260927 18:41:48.508615 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "64a3ed32bd5249d48932c40b1200f7f9"
format_stamp: "Formatted at 2026-09-27 18:41:48 on dist-test-slave-bzzj"
server_key: "c932abb3dd9df525d7b67eb0b815ea4d"
server_key_iv: "77065d30a739308ec872a50f2130567d"
server_key_version: "encryptionkey@0"
I20260927 18:41:48.512355 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 18:41:48.514740 27071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:48.515925 27056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:48.516040 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "64a3ed32bd5249d48932c40b1200f7f9"
format_stamp: "Formatted at 2026-09-27 18:41:48 on dist-test-slave-bzzj"
server_key: "c932abb3dd9df525d7b67eb0b815ea4d"
server_key_iv: "77065d30a739308ec872a50f2130567d"
server_key_version: "encryptionkey@0"
I20260927 18:41:48.516198 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:48.537252 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:48.540514 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:48.540697 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:48.547394 27056 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34421
I20260927 18:41:48.547413 27123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34421 every 8 connection(s)
I20260927 18:41:48.548305 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:48.550174 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27056
I20260927 18:41:48.550319 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:48.550590 6160 external_mini_cluster.cc:1428] Setting key e3188199f7b7df0ffd9c549a923fc067
I20260927 18:41:48.551095 27124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:48.557816 27124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9: Bootstrap starting.
Sep 27 18:41:48 dist-test-slave-bzzj krb5kdc[27043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:41:48.560302 27124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:48.561154 27124 log.cc:824] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:48.563282 27124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9: No bootstrap required, opened a new log
I20260927 18:41:48.565234 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:48.551949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:45609 (local address 127.0.0.1:34421)
0927 18:41:48.552363 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:48.552390 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:48.552431 (+ 41us) server_negotiation.cc:407] Connection header received
0927 18:41:48.553171 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:48.553193 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:48.553628 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:48.554016 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:48.554626 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:48.556063 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:48.556900 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:48.557566 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:48.557714 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:48.560196 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:48.560203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:48.560219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:48.560251 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:48.561952 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:48.562417 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:48.562422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:48.562429 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:48.562558 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:48.562836 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:48.562839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:48.562841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:48.563316 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:48.563524 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:48.563778 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 18:41:48.563997 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":133,"threads_started":1}
I20260927 18:41:48.566918 27124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a3ed32bd5249d48932c40b1200f7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260927 18:41:48.567157 27124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:48.567230 27124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a3ed32bd5249d48932c40b1200f7f9, State: Initialized, Role: FOLLOWER
I20260927 18:41:48.567699 27124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a3ed32bd5249d48932c40b1200f7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260927 18:41:48.567839 27124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:48.567914 27124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:48.568017 27124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:48.568259 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:48.569274 27124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a3ed32bd5249d48932c40b1200f7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260927 18:41:48.569705 27124 leader_election.cc:302] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64a3ed32bd5249d48932c40b1200f7f9; no voters:
I20260927 18:41:48.570127 27124 leader_election.cc:288] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:48.570206 27133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:48.570438 27133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [term 1 LEADER]: Becoming Leader. State: Replica: 64a3ed32bd5249d48932c40b1200f7f9, State: Running, Role: LEADER
I20260927 18:41:48.570878 27133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a3ed32bd5249d48932c40b1200f7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260927 18:41:48.571609 27124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:48.572829 27134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64a3ed32bd5249d48932c40b1200f7f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a3ed32bd5249d48932c40b1200f7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } } }
I20260927 18:41:48.572959 27134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:48.573167 27134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64a3ed32bd5249d48932c40b1200f7f9. Latest consensus state: current_term: 1 leader_uuid: "64a3ed32bd5249d48932c40b1200f7f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a3ed32bd5249d48932c40b1200f7f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } } }
I20260927 18:41:48.573231 27134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:48.573639 27143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:48.577723 27143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:48.581123 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:48.570826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:48485 (local address 127.0.0.1:34421)
0927 18:41:48.570940 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:48.570945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:48.571395 (+ 450us) server_negotiation.cc:407] Connection header received
0927 18:41:48.571518 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:48.571521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:48.571575 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:48.571660 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:48.572525 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:48.573340 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:48.574539 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:48.574861 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:48.574933 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:48.576923 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:48.576928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:48.576931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:48.576959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:48.578714 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:48.579259 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:48.579264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:48.579266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:48.579367 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:48.579735 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:48.579738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:48.579740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:48.580153 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:48.580327 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:48.580837 (+ 510us) server_negotiation.cc:299] Negotiation successful
0927 18:41:48.580944 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:41:48.585072 27143 catalog_manager.cc:1347] Generated new cluster ID: b9346c89a4ff4fac966fc6ea122b82cd
I20260927 18:41:48.585147 27143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:48.596479 27143 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:48.597884 27143 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:48.607640 27143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64a3ed32bd5249d48932c40b1200f7f9: Generated new TSK 0
I20260927 18:41:48.608661 27143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:48.619962 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27056
W20260927 18:41:48.626282 27041 connection.cc:535] client connection to 127.0.0.1:34421 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34421): Connection reset by peer (error 104)
2026-09-27T18:41:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:48 dist-test-slave-bzzj krb5kdc[27155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:48 dist-test-slave-bzzj krb5kdc[27155](info): set up 2 sockets
Sep 27 18:41:48 dist-test-slave-bzzj krb5kdc[27155](info): commencing operation
krb5kdc: starting...
W20260927 18:41:49.579950 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:49 dist-test-slave-bzzj krb5kdc[27155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:49.694793 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:49.800699 27168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:49.800972 27168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:49.801028 27168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:49.805022 27168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:49.805090 27168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:49.805117 27168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:49.805140 27168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:49.805167 27168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:49.810451 27168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:49.811532 27168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:49.812549 27168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:49.820662 27176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:49.820864 27168 server_base.cc:1034] running on GCE node
W20260927 18:41:49.820650 27174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:49.820650 27173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:49.821493 27168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:49.822561 27168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:49.823724 27168 hybrid_clock.cc:648] HybridClock initialized: now 1790534509823683 us; error 54 us; skew 500 ppm
Sep 27 18:41:49 dist-test-slave-bzzj krb5kdc[27155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534509, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:49.826283 27168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:49.827126 27168 webserver.cc:460] Webserver started at http://127.0.0.1:45331/ using document root <none> and password file <none>
I20260927 18:41:49.827620 27168 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:49.827674 27168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:49.827833 27168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:49.829798 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "993d2b06ef224525a611aa4f62bdcffc"
format_stamp: "Formatted at 2026-09-27 18:41:49 on dist-test-slave-bzzj"
server_key: "6e3477c02efab2e7a07bf8f60bfa5f17"
server_key_iv: "a08dd0fa750e6545cf81fe780f2763a0"
server_key_version: "encryptionkey@0"
I20260927 18:41:49.830282 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "993d2b06ef224525a611aa4f62bdcffc"
format_stamp: "Formatted at 2026-09-27 18:41:49 on dist-test-slave-bzzj"
server_key: "6e3477c02efab2e7a07bf8f60bfa5f17"
server_key_iv: "a08dd0fa750e6545cf81fe780f2763a0"
server_key_version: "encryptionkey@0"
I20260927 18:41:49.833758 27168 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:49.835960 27183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:49.837155 27168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:49.837256 27168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "993d2b06ef224525a611aa4f62bdcffc"
format_stamp: "Formatted at 2026-09-27 18:41:49 on dist-test-slave-bzzj"
server_key: "6e3477c02efab2e7a07bf8f60bfa5f17"
server_key_iv: "a08dd0fa750e6545cf81fe780f2763a0"
server_key_version: "encryptionkey@0"
I20260927 18:41:49.837356 27168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:49.887447 27168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:49.890268 27168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:49.890452 27168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:49.897059 27168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45235
I20260927 18:41:49.897076 27235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45235 every 8 connection(s)
I20260927 18:41:49.898027 27168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:49.900695 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:49.901036 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27168
I20260927 18:41:49.901149 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:49.901441 6160 external_mini_cluster.cc:1428] Setting key 441e5dea04d098cd8a51d2dc21d0753d
I20260927 18:41:49.907546 27236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc: Bootstrap starting.
I20260927 18:41:49.910082 27236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:49.910923 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:49.902753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:42557 (local address 127.0.0.1:45235)
0927 18:41:49.903137 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:49.903156 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:49.903198 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:41:49.903899 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:49.903921 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:49.904399 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:49.904753 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:49.905144 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:49.906621 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:49.907484 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:49.908144 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:49.908297 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:49.909058 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:49.909063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:49.909078 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:49.909556 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:49.909618 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:49.909648 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:49.909772 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:49.909876 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 18:41:49.910005 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":89,"threads_started":1}
I20260927 18:41:49.910919 27236 log.cc:824] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:49.913092 27236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc: No bootstrap required, opened a new log
I20260927 18:41:49.913867 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:49.916801 27236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993d2b06ef224525a611aa4f62bdcffc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45235 } }
I20260927 18:41:49.917016 27236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:49.917068 27236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993d2b06ef224525a611aa4f62bdcffc, State: Initialized, Role: FOLLOWER
I20260927 18:41:49.917469 27236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993d2b06ef224525a611aa4f62bdcffc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45235 } }
I20260927 18:41:49.917591 27236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:49.917652 27236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:49.917744 27236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:49.918565 27236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993d2b06ef224525a611aa4f62bdcffc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45235 } }
I20260927 18:41:49.918880 27236 leader_election.cc:302] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993d2b06ef224525a611aa4f62bdcffc; no voters:
I20260927 18:41:49.919152 27236 leader_election.cc:288] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:49.919234 27246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:49.919337 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:49.915707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:50401 (local address 127.0.0.1:45235)
0927 18:41:49.915813 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:49.915816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:49.915901 (+ 85us) server_negotiation.cc:407] Connection header received
0927 18:41:49.915997 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:49.916000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:49.916045 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:49.916188 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:49.916414 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:49.916926 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:49.917736 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:49.918013 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:49.918099 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:49.918829 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:49.918832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:49.918835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:49.918847 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:49.918872 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:49.918882 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:49.918947 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:49.919085 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 18:41:49.919116 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:41:49.919437 27246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [term 1 LEADER]: Becoming Leader. State: Replica: 993d2b06ef224525a611aa4f62bdcffc, State: Running, Role: LEADER
I20260927 18:41:49.919734 27246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993d2b06ef224525a611aa4f62bdcffc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45235 } }
I20260927 18:41:49.920468 27236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:49.920902 27246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993d2b06ef224525a611aa4f62bdcffc. Latest consensus state: current_term: 1 leader_uuid: "993d2b06ef224525a611aa4f62bdcffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993d2b06ef224525a611aa4f62bdcffc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45235 } } }
I20260927 18:41:49.921011 27246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:49.921222 27246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993d2b06ef224525a611aa4f62bdcffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993d2b06ef224525a611aa4f62bdcffc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45235 } } }
I20260927 18:41:49.921367 27246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:49.922135 27251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:49.924815 27251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:49.929945 27251 catalog_manager.cc:1347] Generated new cluster ID: d69e647bf7a143af8ac3664b18ac6bab
I20260927 18:41:49.930009 27251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:49.946554 27251 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:49.948510 27251 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:49.963543 27251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 993d2b06ef224525a611aa4f62bdcffc: Generated new TSK 0
I20260927 18:41:49.964448 27251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:49.973212 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27168
W20260927 18:41:49.978493 27153 connection.cc:535] client connection to 127.0.0.1:45235 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45235): Connection reset by peer (error 104)
2026-09-27T18:41:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:50 dist-test-slave-bzzj krb5kdc[27267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:50 dist-test-slave-bzzj krb5kdc[27267](info): set up 2 sockets
Sep 27 18:41:50 dist-test-slave-bzzj krb5kdc[27267](info): commencing operation
krb5kdc: starting...
W20260927 18:41:50.920030 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:50 dist-test-slave-bzzj krb5kdc[27267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:51.032837 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:51.138844 27280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:51.139113 27280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:51.139156 27280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:51.143157 27280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:51.143234 27280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:51.143253 27280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:51.143267 27280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:51.143282 27280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:51.148674 27280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:51.149739 27280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:51.150746 27280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:51.158979 27288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:51.159018 27285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:51.158993 27286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:51.159246 27280 server_base.cc:1034] running on GCE node
I20260927 18:41:51.159780 27280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:51.160868 27280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:51.162025 27280 hybrid_clock.cc:648] HybridClock initialized: now 1790534511162003 us; error 39 us; skew 500 ppm
Sep 27 18:41:51 dist-test-slave-bzzj krb5kdc[27267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534511, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:51.164625 27280 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:51.165491 27280 webserver.cc:460] Webserver started at http://127.0.0.1:34075/ using document root <none> and password file <none>
I20260927 18:41:51.166013 27280 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:51.166078 27280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:51.166265 27280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:51.168282 27280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "5fd95120fe3c4e5d9894bd5964c41b80"
format_stamp: "Formatted at 2026-09-27 18:41:51 on dist-test-slave-bzzj"
server_key: "e988dbd8e6722a36f99a57953e769e5a"
server_key_iv: "fafabb4120177c26a15329556ec4a6ca"
server_key_version: "encryptionkey@0"
I20260927 18:41:51.168812 27280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "5fd95120fe3c4e5d9894bd5964c41b80"
format_stamp: "Formatted at 2026-09-27 18:41:51 on dist-test-slave-bzzj"
server_key: "e988dbd8e6722a36f99a57953e769e5a"
server_key_iv: "fafabb4120177c26a15329556ec4a6ca"
server_key_version: "encryptionkey@0"
I20260927 18:41:51.172385 27280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:51.174681 27295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:51.175829 27280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:51.175948 27280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "5fd95120fe3c4e5d9894bd5964c41b80"
format_stamp: "Formatted at 2026-09-27 18:41:51 on dist-test-slave-bzzj"
server_key: "e988dbd8e6722a36f99a57953e769e5a"
server_key_iv: "fafabb4120177c26a15329556ec4a6ca"
server_key_version: "encryptionkey@0"
I20260927 18:41:51.176043 27280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:51.180447 27280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:51.183113 27280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:51.183279 27280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:51.189930 27280 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33539
I20260927 18:41:51.189942 27347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33539 every 8 connection(s)
I20260927 18:41:51.190821 27280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:51.193596 27348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:51.198843 27348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80: Bootstrap starting.
I20260927 18:41:51.198872 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27280
I20260927 18:41:51.198997 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:51.199286 6160 external_mini_cluster.cc:1428] Setting key c3a2f1f2cc58001cd3b07dbf145cb470
I20260927 18:41:51.201406 27348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:51.202092 27348 log.cc:824] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:51.204216 27348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80: No bootstrap required, opened a new log
I20260927 18:41:51.204478 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:51.200648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:43685 (local address 127.0.0.1:33539)
0927 18:41:51.201079 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:51.201094 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:51.201129 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:41:51.201791 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:51.201810 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:51.202098 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:51.202422 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:51.203114 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:51.203120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:51.203133 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:51.203144 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:51.203201 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:51.203225 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:51.203326 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:51.203390 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 18:41:51.203556 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260927 18:41:51.207032 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:51.207978 27348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33539 } }
I20260927 18:41:51.208267 27348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:51.208326 27348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd95120fe3c4e5d9894bd5964c41b80, State: Initialized, Role: FOLLOWER
I20260927 18:41:51.208792 27348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33539 } }
I20260927 18:41:51.208956 27348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:51.209030 27348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:51.209131 27348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:51.210178 27348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33539 } }
I20260927 18:41:51.210513 27348 leader_election.cc:302] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fd95120fe3c4e5d9894bd5964c41b80; no voters:
I20260927 18:41:51.210692 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:51.209248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:37841 (local address 127.0.0.1:33539)
0927 18:41:51.209362 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:51.209366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:51.209519 (+ 153us) server_negotiation.cc:407] Connection header received
0927 18:41:51.209617 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:51.209620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:51.209653 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:51.209720 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:51.210340 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:51.210343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:51.210346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:51.210357 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:51.210385 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:51.210398 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:51.210497 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:51.210548 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:41:51.210584 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:41:51.210896 27348 leader_election.cc:288] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:51.210959 27358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:51.211158 27358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [term 1 LEADER]: Becoming Leader. State: Replica: 5fd95120fe3c4e5d9894bd5964c41b80, State: Running, Role: LEADER
I20260927 18:41:51.211503 27358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33539 } }
I20260927 18:41:51.212225 27348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:51.212769 27360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fd95120fe3c4e5d9894bd5964c41b80. Latest consensus state: current_term: 1 leader_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33539 } } }
I20260927 18:41:51.212885 27360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:51.213104 27359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd95120fe3c4e5d9894bd5964c41b80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33539 } } }
I20260927 18:41:51.213249 27359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80 [sys.catalog]: This master's current role is: LEADER
W20260927 18:41:51.216504 27373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:51.216581 27373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:51.216656 27363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:51.217312 27363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:51.222681 27363 catalog_manager.cc:1347] Generated new cluster ID: d4f53a6ada4d484cbec0d4b75e07911f
I20260927 18:41:51.222748 27363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:51.235424 27363 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:51.235775 27363 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:51.243360 27363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fd95120fe3c4e5d9894bd5964c41b80: Generated new TSK 0
I20260927 18:41:51.244196 27363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:51.246618 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27280
2026-09-27T18:41:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:51 dist-test-slave-bzzj krb5kdc[27379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:51 dist-test-slave-bzzj krb5kdc[27379](info): set up 2 sockets
Sep 27 18:41:51 dist-test-slave-bzzj krb5kdc[27379](info): commencing operation
krb5kdc: starting...
W20260927 18:41:52.195964 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:52 dist-test-slave-bzzj krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:52.306587 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:52.413836 27392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:52.414113 27392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:52.414172 27392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:52.418264 27392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:52.418347 27392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:52.418375 27392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:52.418434 27392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:52.418469 27392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:52.424239 27392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:52.425328 27392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:52.426332 27392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:52.435612 27397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:52.435612 27398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:52.435819 27392 server_base.cc:1034] running on GCE node
W20260927 18:41:52.435690 27400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:52.436517 27392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:52.437623 27392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:52.438802 27392 hybrid_clock.cc:648] HybridClock initialized: now 1790534512438775 us; error 40 us; skew 500 ppm
Sep 27 18:41:52 dist-test-slave-bzzj krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:52.448657 27392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:52.449620 27392 webserver.cc:460] Webserver started at http://127.0.0.1:41801/ using document root <none> and password file <none>
I20260927 18:41:52.450176 27392 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:52.450234 27392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:52.450410 27392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:52.452564 27392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "55c391815d58426dad2b3a7cab162b53"
format_stamp: "Formatted at 2026-09-27 18:41:52 on dist-test-slave-bzzj"
server_key: "f285ea0283a7da6c85bc4e6ce024d7e5"
server_key_iv: "6bfdb539a075bf230d363c60e5253d66"
server_key_version: "encryptionkey@0"
I20260927 18:41:52.453081 27392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "55c391815d58426dad2b3a7cab162b53"
format_stamp: "Formatted at 2026-09-27 18:41:52 on dist-test-slave-bzzj"
server_key: "f285ea0283a7da6c85bc4e6ce024d7e5"
server_key_iv: "6bfdb539a075bf230d363c60e5253d66"
server_key_version: "encryptionkey@0"
I20260927 18:41:52.456758 27392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:52.459187 27407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:52.460498 27392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:52.460603 27392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "55c391815d58426dad2b3a7cab162b53"
format_stamp: "Formatted at 2026-09-27 18:41:52 on dist-test-slave-bzzj"
server_key: "f285ea0283a7da6c85bc4e6ce024d7e5"
server_key_iv: "6bfdb539a075bf230d363c60e5253d66"
server_key_version: "encryptionkey@0"
I20260927 18:41:52.460706 27392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:52.488481 27392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:52.491602 27392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:52.491780 27392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:52.499572 27392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43105
I20260927 18:41:52.499624 27459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43105 every 8 connection(s)
I20260927 18:41:52.500599 27392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:52.502625 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27392
I20260927 18:41:52.502741 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:52.503031 6160 external_mini_cluster.cc:1428] Setting key d8afc028a98df046af966446ca0efdcf
I20260927 18:41:52.503942 27460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:52.509419 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53: Bootstrap starting.
Sep 27 18:41:52 dist-test-slave-bzzj krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:41:52.511808 27460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:52.512567 27460 log.cc:824] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:52.514681 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53: No bootstrap required, opened a new log
I20260927 18:41:52.516670 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:52.504468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:45679 (local address 127.0.0.1:43105)
0927 18:41:52.504920 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:52.504935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:52.504970 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:41:52.505601 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:52.505618 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:52.505957 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:52.506251 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:52.506998 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:52.507994 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:52.508842 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:52.509315 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:52.509463 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:52.512032 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:52.512037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:52.512050 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:52.512077 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:52.513906 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:52.514402 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:52.514408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:52.514414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:52.514543 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:52.514842 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:52.514845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:52.514846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:52.515220 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:52.515418 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:52.515718 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 18:41:52.515935 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":142,"threads_started":1}
I20260927 18:41:52.518499 27460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c391815d58426dad2b3a7cab162b53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43105 } }
I20260927 18:41:52.518767 27460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:52.518838 27460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c391815d58426dad2b3a7cab162b53, State: Initialized, Role: FOLLOWER
I20260927 18:41:52.519275 27460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c391815d58426dad2b3a7cab162b53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43105 } }
I20260927 18:41:52.519426 27460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:52.519490 27460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:52.519582 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:52.519586 27460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:52.520485 27460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c391815d58426dad2b3a7cab162b53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43105 } }
I20260927 18:41:52.520807 27460 leader_election.cc:302] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55c391815d58426dad2b3a7cab162b53; no voters:
I20260927 18:41:52.521070 27460 leader_election.cc:288] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:52.521184 27469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:52.521382 27469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [term 1 LEADER]: Becoming Leader. State: Replica: 55c391815d58426dad2b3a7cab162b53, State: Running, Role: LEADER
I20260927 18:41:52.521688 27469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c391815d58426dad2b3a7cab162b53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43105 } }
I20260927 18:41:52.522200 27460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:52.522670 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55c391815d58426dad2b3a7cab162b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c391815d58426dad2b3a7cab162b53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43105 } } }
I20260927 18:41:52.522766 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:52.522936 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55c391815d58426dad2b3a7cab162b53. Latest consensus state: current_term: 1 leader_uuid: "55c391815d58426dad2b3a7cab162b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55c391815d58426dad2b3a7cab162b53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43105 } } }
I20260927 18:41:52.522997 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53 [sys.catalog]: This master's current role is: LEADER
W20260927 18:41:52.526947 27485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:52.527022 27485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:52.527079 27475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:52.527889 27475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:52.531284 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:52.521430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:56617 (local address 127.0.0.1:43105)
0927 18:41:52.521530 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:52.521534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:52.524555 (+ 3021us) server_negotiation.cc:407] Connection header received
0927 18:41:52.524667 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:52.524669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:52.524709 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:52.524787 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:52.525431 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:52.525930 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:52.526705 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:52.526934 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:52.527018 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:52.528388 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:52.528391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:52.528394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:52.528417 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:52.529676 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:52.530124 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:52.530127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:52.530128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:52.530197 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:52.530495 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:52.530498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:52.530499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:52.530655 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:52.530782 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:52.531056 (+ 274us) server_negotiation.cc:299] Negotiation successful
0927 18:41:52.531138 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:41:52.533277 27475 catalog_manager.cc:1347] Generated new cluster ID: cee7fb0a2bdf42fcbd8e955a98507a5e
I20260927 18:41:52.533339 27475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:52.545982 27475 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:52.547169 27475 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:52.553140 27475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55c391815d58426dad2b3a7cab162b53: Generated new TSK 0
I20260927 18:41:52.553921 27475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:52.557224 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27392
W20260927 18:41:52.562340 27377 connection.cc:535] client connection to 127.0.0.1:43105 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43105): Connection reset by peer (error 104)
2026-09-27T18:41:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:52 dist-test-slave-bzzj krb5kdc[27491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:52 dist-test-slave-bzzj krb5kdc[27491](info): set up 2 sockets
Sep 27 18:41:52 dist-test-slave-bzzj krb5kdc[27491](info): commencing operation
krb5kdc: starting...
W20260927 18:41:53.503993 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:53 dist-test-slave-bzzj krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:53.619012 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:53.726902 27504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:53.727165 27504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:53.727207 27504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:53.731168 27504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:53.731240 27504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:53.731258 27504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:53.731273 27504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:53.731287 27504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:53.736515 27504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:53.737507 27504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:53.738456 27504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:53.746565 27512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:53.746565 27509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:53.746592 27510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:53.746667 27504 server_base.cc:1034] running on GCE node
I20260927 18:41:53.747398 27504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:53.748518 27504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:53.749682 27504 hybrid_clock.cc:648] HybridClock initialized: now 1790534513749666 us; error 33 us; skew 500 ppm
Sep 27 18:41:53 dist-test-slave-bzzj krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534513, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:53.752297 27504 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:53.753131 27504 webserver.cc:460] Webserver started at http://127.0.0.1:38909/ using document root <none> and password file <none>
I20260927 18:41:53.753654 27504 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:53.753711 27504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:53.753921 27504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:53.755911 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "a85d612a999b4506bb1cba099b22ae06"
format_stamp: "Formatted at 2026-09-27 18:41:53 on dist-test-slave-bzzj"
server_key: "cbb27577f0c7eb4256c54f3a36bf7714"
server_key_iv: "e56599f14ceda8dce888c00535930db8"
server_key_version: "encryptionkey@0"
I20260927 18:41:53.756479 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "a85d612a999b4506bb1cba099b22ae06"
format_stamp: "Formatted at 2026-09-27 18:41:53 on dist-test-slave-bzzj"
server_key: "cbb27577f0c7eb4256c54f3a36bf7714"
server_key_iv: "e56599f14ceda8dce888c00535930db8"
server_key_version: "encryptionkey@0"
I20260927 18:41:53.760047 27504 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:53.762354 27519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:53.763607 27504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:53.763720 27504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "a85d612a999b4506bb1cba099b22ae06"
format_stamp: "Formatted at 2026-09-27 18:41:53 on dist-test-slave-bzzj"
server_key: "cbb27577f0c7eb4256c54f3a36bf7714"
server_key_iv: "e56599f14ceda8dce888c00535930db8"
server_key_version: "encryptionkey@0"
I20260927 18:41:53.763821 27504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:53.778682 27504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:53.781957 27504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:53.782140 27504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:53.789060 27504 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44177
I20260927 18:41:53.789075 27571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44177 every 8 connection(s)
I20260927 18:41:53.790120 27504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:53.792873 27572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:53.794835 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27504
I20260927 18:41:53.794986 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:53.795289 6160 external_mini_cluster.cc:1428] Setting key e1985f5ddaedc1687cef65101c955d3e
I20260927 18:41:53.799729 27572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06: Bootstrap starting.
I20260927 18:41:53.802414 27572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:53.803150 27572 log.cc:824] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:53.804313 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:53.796609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:60539 (local address 127.0.0.1:44177)
0927 18:41:53.796997 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:53.797012 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:53.797048 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:41:53.797703 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:53.797720 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:53.798072 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:53.798390 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:53.798820 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:53.799822 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:53.801040 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:53.801515 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:53.801633 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:53.802492 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:53.802497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:53.802510 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:53.802926 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:53.802980 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:53.803004 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:53.803103 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:53.803307 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 18:41:53.803426 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260927 18:41:53.805176 27572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06: No bootstrap required, opened a new log
I20260927 18:41:53.807273 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:53.809106 27572 raft_consensus.cc:348] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85d612a999b4506bb1cba099b22ae06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260927 18:41:53.809386 27572 raft_consensus.cc:374] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:53.809491 27572 raft_consensus.cc:729] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85d612a999b4506bb1cba099b22ae06, State: Initialized, Role: FOLLOWER
I20260927 18:41:53.809912 27572 consensus_queue.cc:260] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85d612a999b4506bb1cba099b22ae06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260927 18:41:53.810050 27572 raft_consensus.cc:388] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:53.810113 27572 raft_consensus.cc:482] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:53.810207 27572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:53.811062 27572 raft_consensus.cc:504] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85d612a999b4506bb1cba099b22ae06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260927 18:41:53.811373 27572 leader_election.cc:302] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a85d612a999b4506bb1cba099b22ae06; no voters:
I20260927 18:41:53.811662 27572 leader_election.cc:288] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:53.811743 27582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:53.811950 27582 raft_consensus.cc:686] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [term 1 LEADER]: Becoming Leader. State: Replica: a85d612a999b4506bb1cba099b22ae06, State: Running, Role: LEADER
I20260927 18:41:53.812301 27582 consensus_queue.cc:237] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85d612a999b4506bb1cba099b22ae06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260927 18:41:53.813032 27572 sys_catalog.cc:564] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:53.813158 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:53.809183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:55405 (local address 127.0.0.1:44177)
0927 18:41:53.809286 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:53.809290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:53.809440 (+ 150us) server_negotiation.cc:407] Connection header received
0927 18:41:53.809562 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:53.809565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:53.809612 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:53.809673 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:53.809943 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:53.810552 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:53.811365 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:53.811644 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:53.811726 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:53.812682 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:53.812686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:53.812689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:53.812702 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:53.812728 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:53.812746 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:53.812816 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:53.812950 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 18:41:53.812981 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:41:53.813541 27582 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a85d612a999b4506bb1cba099b22ae06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85d612a999b4506bb1cba099b22ae06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } } }
I20260927 18:41:53.813717 27582 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:53.813936 27583 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a85d612a999b4506bb1cba099b22ae06. Latest consensus state: current_term: 1 leader_uuid: "a85d612a999b4506bb1cba099b22ae06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85d612a999b4506bb1cba099b22ae06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } } }
I20260927 18:41:53.814076 27583 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:53.814206 27588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:53.817381 27588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:53.822249 27588 catalog_manager.cc:1347] Generated new cluster ID: c83457711521453bbb4ed7359c965694
I20260927 18:41:53.822321 27588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:53.834920 27588 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:53.836233 27588 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:53.845118 27588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a85d612a999b4506bb1cba099b22ae06: Generated new TSK 0
I20260927 18:41:53.845840 27588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:53.852308 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27504
W20260927 18:41:53.856957 27489 connection.cc:535] client connection to 127.0.0.1:44177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44177): Connection reset by peer (error 104)
2026-09-27T18:41:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:53 dist-test-slave-bzzj krb5kdc[27603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:53 dist-test-slave-bzzj krb5kdc[27603](info): set up 2 sockets
Sep 27 18:41:53 dist-test-slave-bzzj krb5kdc[27603](info): commencing operation
krb5kdc: starting...
W20260927 18:41:54.795897 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:54 dist-test-slave-bzzj krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:41:54.906986 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42697
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:55.013752 27616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:55.014070 27616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:55.014112 27616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:55.018168 27616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:55.018234 27616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:55.018251 27616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:55.018267 27616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:55.018280 27616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:55.023535 27616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42697
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:55.024595 27616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:55.025605 27616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:55.034264 27621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:55.034327 27622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:55.034302 27616 server_base.cc:1034] running on GCE node
W20260927 18:41:55.034308 27624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:55.034960 27616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:55.035979 27616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:55.037127 27616 hybrid_clock.cc:648] HybridClock initialized: now 1790534515037096 us; error 41 us; skew 500 ppm
Sep 27 18:41:55 dist-test-slave-bzzj krb5kdc[27603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534515, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:55.039908 27616 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:41:55.040751 27616 webserver.cc:460] Webserver started at http://127.0.0.1:36813/ using document root <none> and password file <none>
I20260927 18:41:55.041257 27616 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:55.041312 27616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:55.041472 27616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:55.043437 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "89a69866a1d24179b2ea3dc65c902eb2"
format_stamp: "Formatted at 2026-09-27 18:41:55 on dist-test-slave-bzzj"
server_key: "1e5b551407356740c51e1d589c34564a"
server_key_iv: "cc21eee0c2d1d2fdc4862f9879ca32ca"
server_key_version: "encryptionkey@0"
I20260927 18:41:55.043918 27616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "89a69866a1d24179b2ea3dc65c902eb2"
format_stamp: "Formatted at 2026-09-27 18:41:55 on dist-test-slave-bzzj"
server_key: "1e5b551407356740c51e1d589c34564a"
server_key_iv: "cc21eee0c2d1d2fdc4862f9879ca32ca"
server_key_version: "encryptionkey@0"
I20260927 18:41:55.047408 27616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260927 18:41:55.049701 27631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:55.050879 27616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:41:55.050987 27616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "89a69866a1d24179b2ea3dc65c902eb2"
format_stamp: "Formatted at 2026-09-27 18:41:55 on dist-test-slave-bzzj"
server_key: "1e5b551407356740c51e1d589c34564a"
server_key_iv: "cc21eee0c2d1d2fdc4862f9879ca32ca"
server_key_version: "encryptionkey@0"
I20260927 18:41:55.051077 27616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:55.055480 27616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:55.058347 27616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:55.058509 27616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:55.064929 27616 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42697
I20260927 18:41:55.064944 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42697 every 8 connection(s)
I20260927 18:41:55.065866 27616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:55.068600 27684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:55.072638 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27616
I20260927 18:41:55.072773 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:55.073057 6160 external_mini_cluster.cc:1428] Setting key 34717f3e2d1f4d6aef343772b61e7c60
I20260927 18:41:55.074421 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2: Bootstrap starting.
I20260927 18:41:55.076802 27684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:55.077598 27684 log.cc:824] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:55.077960 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:55.074357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:44839 (local address 127.0.0.1:42697)
0927 18:41:55.074751 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:55.074765 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:55.074801 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:41:55.075435 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:55.075451 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:55.075726 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:55.076002 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:55.076773 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:55.076778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:55.076789 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:55.076798 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:55.076847 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:55.076865 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:55.076962 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:55.077078 (+ 116us) server_negotiation.cc:299] Negotiation successful
0927 18:41:55.077221 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260927 18:41:55.079347 27684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2: No bootstrap required, opened a new log
I20260927 18:41:55.080497 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:55.083024 27684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a69866a1d24179b2ea3dc65c902eb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42697 } }
I20260927 18:41:55.083236 27684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:55.083290 27684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a69866a1d24179b2ea3dc65c902eb2, State: Initialized, Role: FOLLOWER
I20260927 18:41:55.083721 27684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a69866a1d24179b2ea3dc65c902eb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42697 } }
I20260927 18:41:55.083871 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:55.082347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:41547 (local address 127.0.0.1:42697)
0927 18:41:55.082455 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:55.082459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:55.082603 (+ 144us) server_negotiation.cc:407] Connection header received
0927 18:41:55.082723 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:55.082726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:55.082758 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:55.082815 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:55.083437 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:55.083441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:55.083443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:55.083454 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:55.083480 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:55.083492 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:55.083568 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:55.083673 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 18:41:55.083703 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:41:55.083887 27684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:55.083976 27684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:55.084070 27684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:55.085037 27684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a69866a1d24179b2ea3dc65c902eb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42697 } }
I20260927 18:41:55.085338 27684 leader_election.cc:302] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89a69866a1d24179b2ea3dc65c902eb2; no voters:
I20260927 18:41:55.085614 27684 leader_election.cc:288] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:55.085695 27694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:55.085886 27694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [term 1 LEADER]: Becoming Leader. State: Replica: 89a69866a1d24179b2ea3dc65c902eb2, State: Running, Role: LEADER
I20260927 18:41:55.086200 27694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a69866a1d24179b2ea3dc65c902eb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42697 } }
I20260927 18:41:55.086746 27684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:55.087819 27695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89a69866a1d24179b2ea3dc65c902eb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a69866a1d24179b2ea3dc65c902eb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42697 } } }
I20260927 18:41:55.087857 27696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89a69866a1d24179b2ea3dc65c902eb2. Latest consensus state: current_term: 1 leader_uuid: "89a69866a1d24179b2ea3dc65c902eb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a69866a1d24179b2ea3dc65c902eb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42697 } } }
I20260927 18:41:55.088023 27696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:55.088013 27695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:55.088868 27704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:55.091355 27704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:55.096705 27704 catalog_manager.cc:1347] Generated new cluster ID: 995700f0408b4fa1a833251d78225267
I20260927 18:41:55.096784 27704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:55.115753 27704 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:55.116065 27704 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:55.124747 27704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89a69866a1d24179b2ea3dc65c902eb2: Generated new TSK 0
I20260927 18:41:55.125420 27704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:55.131373 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27616
2026-09-27T18:41:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:55 dist-test-slave-bzzj krb5kdc[27715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:55 dist-test-slave-bzzj krb5kdc[27715](info): set up 2 sockets
Sep 27 18:41:55 dist-test-slave-bzzj krb5kdc[27715](info): commencing operation
krb5kdc: starting...
W20260927 18:41:56.083945 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:56 dist-test-slave-bzzj krb5kdc[27715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:41:56.196146 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38395
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:38395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:56.303601 27728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:56.303916 27728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:56.303969 27728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:56.307934 27728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:56.308002 27728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:56.308022 27728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:56.308037 27728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:56.308054 27728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:56.313349 27728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:38395
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:56.314409 27728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:56.315397 27728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:56.323683 27734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:56.323693 27733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:56.323755 27736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:56.324059 27728 server_base.cc:1034] running on GCE node
I20260927 18:41:56.324607 27728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:56.325713 27728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:56.326967 27728 hybrid_clock.cc:648] HybridClock initialized: now 1790534516326936 us; error 40 us; skew 500 ppm
Sep 27 18:41:56 dist-test-slave-bzzj krb5kdc[27715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534516, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:56.329630 27728 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:56.330539 27728 webserver.cc:460] Webserver started at http://127.6.4.62:33239/ using document root <none> and password file <none>
I20260927 18:41:56.331073 27728 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:56.331137 27728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:56.331341 27728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:56.333477 27728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "7b70dbf598a74456bda5767a52dd4b4b"
format_stamp: "Formatted at 2026-09-27 18:41:56 on dist-test-slave-bzzj"
server_key: "13f13a2367217ce98d7a07378e72e064"
server_key_iv: "792f9b78851af61342644e5cce6e0386"
server_key_version: "encryptionkey@0"
I20260927 18:41:56.334033 27728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "7b70dbf598a74456bda5767a52dd4b4b"
format_stamp: "Formatted at 2026-09-27 18:41:56 on dist-test-slave-bzzj"
server_key: "13f13a2367217ce98d7a07378e72e064"
server_key_iv: "792f9b78851af61342644e5cce6e0386"
server_key_version: "encryptionkey@0"
I20260927 18:41:56.337663 27728 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:41:56.339917 27743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:56.341149 27728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:56.341287 27728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "7b70dbf598a74456bda5767a52dd4b4b"
format_stamp: "Formatted at 2026-09-27 18:41:56 on dist-test-slave-bzzj"
server_key: "13f13a2367217ce98d7a07378e72e064"
server_key_iv: "792f9b78851af61342644e5cce6e0386"
server_key_version: "encryptionkey@0"
I20260927 18:41:56.341388 27728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:56.371990 27728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:56.375741 27728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:56.375950 27728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:56.384603 27728 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:38395
I20260927 18:41:56.384622 27795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:38395 every 8 connection(s)
I20260927 18:41:56.385771 27728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:56.388793 27796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:56.392509 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27728
I20260927 18:41:56.392642 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:56.392999 6160 external_mini_cluster.cc:1428] Setting key 39db10094d0b56c3a7502d1da458ca4e
I20260927 18:41:56.395994 27796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b: Bootstrap starting.
I20260927 18:41:56.398933 27796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:56.399883 27796 log.cc:824] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:56.402245 27796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b: No bootstrap required, opened a new log
Sep 27 18:41:56 dist-test-slave-bzzj krb5kdc[27715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:41:56.406468 27796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b70dbf598a74456bda5767a52dd4b4b" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38395 } }
I20260927 18:41:56.406767 27796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:56.406824 27796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b70dbf598a74456bda5767a52dd4b4b, State: Initialized, Role: FOLLOWER
I20260927 18:41:56.407352 27796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b70dbf598a74456bda5767a52dd4b4b" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38395 } }
I20260927 18:41:56.407517 27796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:56.407579 27796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:56.407670 27796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:56.408875 27796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b70dbf598a74456bda5767a52dd4b4b" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38395 } }
I20260927 18:41:56.408941 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:56.394497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58997 (local address 127.6.4.62:38395)
0927 18:41:56.394918 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:56.394935 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:56.394974 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:41:56.395621 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:56.395643 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:56.396116 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:56.396482 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:56.397761 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:56.398754 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:56.399902 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:56.400630 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:56.400798 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:56.403573 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:56.403580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:56.403593 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:56.403620 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:56.405667 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:56.406164 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:56.406169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:56.406174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:56.406289 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:56.406596 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:56.406598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:56.406600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:56.407048 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:56.407311 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:56.407601 (+ 290us) server_negotiation.cc:299] Negotiation successful
0927 18:41:56.407963 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260927 18:41:56.409260 27796 leader_election.cc:302] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b70dbf598a74456bda5767a52dd4b4b; no voters:
I20260927 18:41:56.409634 27796 leader_election.cc:288] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:56.409766 27801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:56.409988 27801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [term 1 LEADER]: Becoming Leader. State: Replica: 7b70dbf598a74456bda5767a52dd4b4b, State: Running, Role: LEADER
I20260927 18:41:56.410338 27801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b70dbf598a74456bda5767a52dd4b4b" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38395 } }
I20260927 18:41:56.411046 27796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:56.411995 27802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b70dbf598a74456bda5767a52dd4b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b70dbf598a74456bda5767a52dd4b4b" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38395 } } }
I20260927 18:41:56.412139 27802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:56.412204 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:56.412505 27803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b70dbf598a74456bda5767a52dd4b4b. Latest consensus state: current_term: 1 leader_uuid: "7b70dbf598a74456bda5767a52dd4b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b70dbf598a74456bda5767a52dd4b4b" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38395 } } }
I20260927 18:41:56.412628 27803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b [sys.catalog]: This master's current role is: LEADER
W20260927 18:41:56.415637 27816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:41:56.415701 27816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:41:56.415822 27810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:56.416698 27810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:56.422531 27810 catalog_manager.cc:1347] Generated new cluster ID: b1baa6c635da4fbabac1f8016b1c0479
I20260927 18:41:56.422585 27810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:56.425882 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:56.416123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52029 (local address 127.6.4.62:38395)
0927 18:41:56.416226 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:56.416230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:56.416484 (+ 254us) server_negotiation.cc:407] Connection header received
0927 18:41:56.416650 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:56.416653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:56.416695 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:56.416765 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:56.417421 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:56.417933 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:56.418779 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:56.419139 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:56.419256 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:56.420811 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:56.420815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:56.420818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:41:56.420843 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:56.424042 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:56.424694 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:56.424697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:56.424699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:56.424780 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:41:56.425074 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:41:56.425077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:41:56.425079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:41:56.425263 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:56.425389 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:56.425648 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 18:41:56.425735 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:56.429337 27810 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:56.430142 27810 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:56.441589 27810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b70dbf598a74456bda5767a52dd4b4b: Generated new TSK 0
I20260927 18:41:56.442267 27810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:56.447993 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27728
W20260927 18:41:56.453462 27713 connection.cc:535] client connection to 127.6.4.62:38395 recv error: Network error: failed to read from TLS socket (remote: 127.6.4.62:38395): Connection reset by peer (error 104)
2026-09-27T18:41:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:56 dist-test-slave-bzzj krb5kdc[27827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:56 dist-test-slave-bzzj krb5kdc[27827](info): set up 2 sockets
Sep 27 18:41:56 dist-test-slave-bzzj krb5kdc[27827](info): commencing operation
krb5kdc: starting...
W20260927 18:41:57.395929 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:57 dist-test-slave-bzzj krb5kdc[27827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:41:57.507122 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38363
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:38363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:57.615026 27840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:57.615374 27840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:57.615420 27840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:57.619696 27840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:57.619784 27840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:57.619803 27840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:57.619820 27840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:57.619841 27840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:57.625787 27840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:38363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:38363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:57.627115 27840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:57.628378 27840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:41:57.638221 27840 server_base.cc:1034] running on GCE node
W20260927 18:41:57.638288 27846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:57.638228 27848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:57.638195 27845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:57.639101 27840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:57.640522 27840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:57.641692 27840 hybrid_clock.cc:648] HybridClock initialized: now 1790534517641668 us; error 37 us; skew 500 ppm
Sep 27 18:41:57 dist-test-slave-bzzj krb5kdc[27827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534517, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:57.649238 27840 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:57.650223 27840 webserver.cc:460] Webserver started at http://127.6.4.62:38225/ using document root <none> and password file <none>
I20260927 18:41:57.650779 27840 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:57.650844 27840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:57.651049 27840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:57.653182 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "5b912ac3c353479f9f2a69355df1745c"
format_stamp: "Formatted at 2026-09-27 18:41:57 on dist-test-slave-bzzj"
server_key: "f26ce387aaabc5cc62f6e3b497504116"
server_key_iv: "710578b29b542690ee1f666e2c43cd78"
server_key_version: "encryptionkey@0"
I20260927 18:41:57.653699 27840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "5b912ac3c353479f9f2a69355df1745c"
format_stamp: "Formatted at 2026-09-27 18:41:57 on dist-test-slave-bzzj"
server_key: "f26ce387aaabc5cc62f6e3b497504116"
server_key_iv: "710578b29b542690ee1f666e2c43cd78"
server_key_version: "encryptionkey@0"
I20260927 18:41:57.657323 27840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:41:57.659631 27855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:57.660851 27840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:41:57.660974 27840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "5b912ac3c353479f9f2a69355df1745c"
format_stamp: "Formatted at 2026-09-27 18:41:57 on dist-test-slave-bzzj"
server_key: "f26ce387aaabc5cc62f6e3b497504116"
server_key_iv: "710578b29b542690ee1f666e2c43cd78"
server_key_version: "encryptionkey@0"
I20260927 18:41:57.661074 27840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:57.673558 27840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:57.676353 27840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:57.676545 27840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:57.683054 27840 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:38363
I20260927 18:41:57.683092 27907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:38363 every 8 connection(s)
I20260927 18:41:57.684000 27840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:57.686631 27908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:57.691871 27908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c: Bootstrap starting.
I20260927 18:41:57.693176 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27840
I20260927 18:41:57.693288 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:57.693528 6160 external_mini_cluster.cc:1428] Setting key d846c9ad8081efe648dcc99ebd7a6b3c
I20260927 18:41:57.694574 27908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:57.695279 27908 log.cc:824] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:57.697278 27908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c: No bootstrap required, opened a new log
I20260927 18:41:57.701272 27908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b912ac3c353479f9f2a69355df1745c" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38363 } }
I20260927 18:41:57.701539 27908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:57.701601 27908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b912ac3c353479f9f2a69355df1745c, State: Initialized, Role: FOLLOWER
I20260927 18:41:57.702013 27908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b912ac3c353479f9f2a69355df1745c" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38363 } }
I20260927 18:41:57.702143 27908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:57.702205 27908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:57.702297 27908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:57.702342 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:57.694592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58751 (local address 127.6.4.62:38363)
0927 18:41:57.695028 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:57.695044 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:57.695076 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:41:57.695798 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:57.695822 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:57.696267 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:57.696546 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:57.696908 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:57.697880 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:57.699061 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:57.699537 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:57.699646 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:57.700560 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:57.700564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:57.700577 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:57.700973 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:57.701030 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:57.701051 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:57.701146 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:57.701330 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:41:57.701463 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":125,"threads_started":1}
I20260927 18:41:57.703367 27908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b912ac3c353479f9f2a69355df1745c" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38363 } }
I20260927 18:41:57.703685 27908 leader_election.cc:302] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b912ac3c353479f9f2a69355df1745c; no voters:
I20260927 18:41:57.703972 27908 leader_election.cc:288] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:57.704069 27913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:57.704523 27913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [term 1 LEADER]: Becoming Leader. State: Replica: 5b912ac3c353479f9f2a69355df1745c, State: Running, Role: LEADER
I20260927 18:41:57.704835 27913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b912ac3c353479f9f2a69355df1745c" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38363 } }
I20260927 18:41:57.705231 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:57.705235 27908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:57.705924 27913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b912ac3c353479f9f2a69355df1745c. Latest consensus state: current_term: 1 leader_uuid: "5b912ac3c353479f9f2a69355df1745c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b912ac3c353479f9f2a69355df1745c" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38363 } } }
I20260927 18:41:57.706054 27913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:57.706422 27922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:57.707441 27914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b912ac3c353479f9f2a69355df1745c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b912ac3c353479f9f2a69355df1745c" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 38363 } } }
I20260927 18:41:57.707543 27914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:57.709331 27922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:57.714995 27922 catalog_manager.cc:1347] Generated new cluster ID: 83d4773b6295471582aa9a425eaff8e4
I20260927 18:41:57.715121 27922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:57.716905 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:57.712616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37811 (local address 127.6.4.62:38363)
0927 18:41:57.712722 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:57.712727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:57.712943 (+ 216us) server_negotiation.cc:407] Connection header received
0927 18:41:57.713096 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:57.713099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:57.713152 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:57.713243 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:57.713593 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:57.714120 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:57.715009 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:41:57.715324 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:41:57.715407 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:41:57.716481 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:57.716485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:57.716488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:57.716502 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:57.716531 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:57.716544 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:57.716637 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:57.716727 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 18:41:57.716762 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:41:57.726541 27922 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:57.727480 27922 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:57.733937 27922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b912ac3c353479f9f2a69355df1745c: Generated new TSK 0
I20260927 18:41:57.734628 27922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:57.743757 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27840
W20260927 18:41:57.748270 27825 connection.cc:535] client connection to 127.6.4.62:38363 recv error: Network error: failed to read from TLS socket (remote: 127.6.4.62:38363): Connection reset by peer (error 104)
2026-09-27T18:41:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:57 dist-test-slave-bzzj krb5kdc[27939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:57 dist-test-slave-bzzj krb5kdc[27939](info): set up 2 sockets
Sep 27 18:41:57 dist-test-slave-bzzj krb5kdc[27939](info): commencing operation
krb5kdc: starting...
W20260927 18:41:58.696031 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:41:58 dist-test-slave-bzzj krb5kdc[27939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:41:58.815466 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45635
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:45635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:41:58.923884 27952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:41:58.924176 27952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:41:58.924219 27952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:41:58.928206 27952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:41:58.928268 27952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:41:58.928285 27952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:41:58.928300 27952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:41:58.928314 27952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:41:58.935144 27952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:45635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:45635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:41:58.936496 27952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:41:58.937748 27952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:41:58.946121 27960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:58.946121 27958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:41:58.946491 27957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:41:58.946687 27952 server_base.cc:1034] running on GCE node
I20260927 18:41:58.947363 27952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:41:58.948486 27952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:41:58.949637 27952 hybrid_clock.cc:648] HybridClock initialized: now 1790534518949621 us; error 29 us; skew 500 ppm
Sep 27 18:41:58 dist-test-slave-bzzj krb5kdc[27939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534518, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:41:58.952255 27952 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:41:58.953120 27952 webserver.cc:460] Webserver started at http://127.6.4.62:37053/ using document root <none> and password file <none>
I20260927 18:41:58.953642 27952 fs_manager.cc:359] Metadata directory not provided
I20260927 18:41:58.953696 27952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:41:58.953905 27952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:41:58.955879 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "ac4b882a19e2406a94d7475c2617f718"
format_stamp: "Formatted at 2026-09-27 18:41:58 on dist-test-slave-bzzj"
server_key: "e1e52df9726653e739dbd0fe49d860ea"
server_key_iv: "4fad55c8a2cad86002cd76a2d3afb844"
server_key_version: "encryptionkey@0"
I20260927 18:41:58.956437 27952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "ac4b882a19e2406a94d7475c2617f718"
format_stamp: "Formatted at 2026-09-27 18:41:58 on dist-test-slave-bzzj"
server_key: "e1e52df9726653e739dbd0fe49d860ea"
server_key_iv: "4fad55c8a2cad86002cd76a2d3afb844"
server_key_version: "encryptionkey@0"
I20260927 18:41:58.959956 27952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 18:41:58.962385 27967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:41:58.963569 27952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:41:58.963680 27952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "ac4b882a19e2406a94d7475c2617f718"
format_stamp: "Formatted at 2026-09-27 18:41:58 on dist-test-slave-bzzj"
server_key: "e1e52df9726653e739dbd0fe49d860ea"
server_key_iv: "4fad55c8a2cad86002cd76a2d3afb844"
server_key_version: "encryptionkey@0"
I20260927 18:41:58.963773 27952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:41:58.968152 27952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:41:58.971041 27952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:41:58.971196 27952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:41:58.977679 27952 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:45635
I20260927 18:41:58.977691 28019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:45635 every 8 connection(s)
I20260927 18:41:58.978576 27952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:41:58.981276 28020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:41:58.981943 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 27952
I20260927 18:41:58.982061 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:41:58.982314 6160 external_mini_cluster.cc:1428] Setting key cbcf07d3584c79cd13f1fad463f24ac0
I20260927 18:41:58.987589 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:58.983595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51753 (local address 127.6.4.62:45635)
0927 18:41:58.984004 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:58.984023 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:58.984069 (+ 46us) server_negotiation.cc:407] Connection header received
0927 18:41:58.984879 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:58.984901 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:58.985254 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:58.985612 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:58.986236 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:58.986241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:58.986256 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:58.986266 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:58.986332 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:58.986357 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:58.986497 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:58.986567 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 18:41:58.986718 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260927 18:41:58.987777 28020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718: Bootstrap starting.
I20260927 18:41:58.990243 28020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718: Neither blocks nor log segments found. Creating new log.
I20260927 18:41:58.990345 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:41:58.991091 28020 log.cc:824] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718: Log is configured to *not* fsync() on all Append() calls
I20260927 18:41:58.993251 28020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718: No bootstrap required, opened a new log
I20260927 18:41:58.993970 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:41:58.992515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51939 (local address 127.6.4.62:45635)
0927 18:41:58.992621 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:41:58.992625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:41:58.992754 (+ 129us) server_negotiation.cc:407] Connection header received
0927 18:41:58.992840 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:41:58.992842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:41:58.992874 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:41:58.992956 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:41:58.993627 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:41:58.993630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:41:58.993633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:41:58.993644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:41:58.993671 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:41:58.993683 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:41:58.993777 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:41:58.993827 (+ 50us) server_negotiation.cc:299] Negotiation successful
0927 18:41:58.993861 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:41:58.996995 28020 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b882a19e2406a94d7475c2617f718" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45635 } }
I20260927 18:41:58.997217 28020 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:41:58.997273 28020 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4b882a19e2406a94d7475c2617f718, State: Initialized, Role: FOLLOWER
I20260927 18:41:58.997727 28020 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b882a19e2406a94d7475c2617f718" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45635 } }
I20260927 18:41:58.997900 28020 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:41:58.997965 28020 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:41:58.998061 28020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:41:58.998919 28020 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b882a19e2406a94d7475c2617f718" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45635 } }
I20260927 18:41:58.999233 28020 leader_election.cc:302] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac4b882a19e2406a94d7475c2617f718; no voters:
I20260927 18:41:58.999518 28020 leader_election.cc:288] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:41:58.999608 28030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:41:58.999816 28030 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [term 1 LEADER]: Becoming Leader. State: Replica: ac4b882a19e2406a94d7475c2617f718, State: Running, Role: LEADER
I20260927 18:41:59.000120 28030 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b882a19e2406a94d7475c2617f718" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45635 } }
I20260927 18:41:59.000723 28020 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:41:59.001713 28032 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac4b882a19e2406a94d7475c2617f718. Latest consensus state: current_term: 1 leader_uuid: "ac4b882a19e2406a94d7475c2617f718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b882a19e2406a94d7475c2617f718" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45635 } } }
I20260927 18:41:59.001844 28032 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:59.002046 28031 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac4b882a19e2406a94d7475c2617f718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b882a19e2406a94d7475c2617f718" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 45635 } } }
I20260927 18:41:59.002133 28031 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718 [sys.catalog]: This master's current role is: LEADER
I20260927 18:41:59.002276 28039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:41:59.005395 28039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:41:59.011214 28039 catalog_manager.cc:1347] Generated new cluster ID: dba15c8a84674bbab4e9c74c57415dbc
I20260927 18:41:59.011286 28039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:41:59.026247 28039 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:41:59.026551 28039 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:41:59.042210 28039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac4b882a19e2406a94d7475c2617f718: Generated new TSK 0
I20260927 18:41:59.042917 28039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:41:59.058660 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 27952
2026-09-27T18:41:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:41:59 dist-test-slave-bzzj krb5kdc[28051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:41:59 dist-test-slave-bzzj krb5kdc[28051](info): set up 2 sockets
Sep 27 18:41:59 dist-test-slave-bzzj krb5kdc[28051](info): commencing operation
krb5kdc: starting...
W20260927 18:42:00.008509 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:42:00 dist-test-slave-bzzj krb5kdc[28051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:42:00.127188 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39647
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:39835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:39647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:00.233953 28064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:00.234232 28064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:00.234297 28064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:00.238300 28064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:00.238370 28064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:00.238397 28064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:00.238417 28064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:00.238440 28064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:00.243685 28064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:39835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:39647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:39647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:00.244761 28064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:00.245731 28064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:00.253813 28070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:00.253813 28069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:00.253813 28072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:00.253892 28064 server_base.cc:1034] running on GCE node
I20260927 18:42:00.254539 28064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:00.255659 28064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:00.256896 28064 hybrid_clock.cc:648] HybridClock initialized: now 1790534520256873 us; error 32 us; skew 500 ppm
Sep 27 18:42:00 dist-test-slave-bzzj krb5kdc[28051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534520, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:00.259547 28064 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:42:00.260489 28064 webserver.cc:460] Webserver started at http://127.6.4.62:37793/ using document root <none> and password file <none>
I20260927 18:42:00.261018 28064 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:00.261081 28064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:00.261272 28064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:00.263242 28064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "8485dc5846ac49368eff9aefb0e22066"
format_stamp: "Formatted at 2026-09-27 18:42:00 on dist-test-slave-bzzj"
server_key: "3fda45f1db793e55f519eba1ee7baec9"
server_key_iv: "7ad496a480e8f2344c9800ef23e6c95f"
server_key_version: "encryptionkey@0"
I20260927 18:42:00.263746 28064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "8485dc5846ac49368eff9aefb0e22066"
format_stamp: "Formatted at 2026-09-27 18:42:00 on dist-test-slave-bzzj"
server_key: "3fda45f1db793e55f519eba1ee7baec9"
server_key_iv: "7ad496a480e8f2344c9800ef23e6c95f"
server_key_version: "encryptionkey@0"
I20260927 18:42:00.267283 28064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260927 18:42:00.269647 28079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:00.270807 28064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:42:00.270921 28064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "8485dc5846ac49368eff9aefb0e22066"
format_stamp: "Formatted at 2026-09-27 18:42:00 on dist-test-slave-bzzj"
server_key: "3fda45f1db793e55f519eba1ee7baec9"
server_key_iv: "7ad496a480e8f2344c9800ef23e6c95f"
server_key_version: "encryptionkey@0"
I20260927 18:42:00.271034 28064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:00.307108 28064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:00.310215 28064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:00.310405 28064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:00.317108 28064 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:39647
I20260927 18:42:00.317134 28131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:39647 every 8 connection(s)
I20260927 18:42:00.318024 28064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:00.320720 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:00.323448 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28064
I20260927 18:42:00.323589 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:42:00.323884 6160 external_mini_cluster.cc:1428] Setting key 15f06fdbf153147fdf33c18bc45184e3
I20260927 18:42:00.326447 28132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066: Bootstrap starting.
I20260927 18:42:00.328886 28132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:00.329782 28132 log.cc:824] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:00.331929 28132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066: No bootstrap required, opened a new log
Sep 27 18:42:00 dist-test-slave-bzzj krb5kdc[28051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:00.335321 28132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8485dc5846ac49368eff9aefb0e22066" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39647 } }
I20260927 18:42:00.335558 28132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:00.335615 28132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8485dc5846ac49368eff9aefb0e22066, State: Initialized, Role: FOLLOWER
I20260927 18:42:00.336133 28132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8485dc5846ac49368eff9aefb0e22066" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39647 } }
I20260927 18:42:00.336287 28132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:00.336354 28132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:00.336448 28132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:00.337319 28132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8485dc5846ac49368eff9aefb0e22066" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39647 } }
I20260927 18:42:00.337682 28132 leader_election.cc:302] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8485dc5846ac49368eff9aefb0e22066; no voters:
I20260927 18:42:00.337996 28132 leader_election.cc:288] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:42:00.338132 28137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:00.338330 28137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [term 1 LEADER]: Becoming Leader. State: Replica: 8485dc5846ac49368eff9aefb0e22066, State: Running, Role: LEADER
I20260927 18:42:00.338631 28137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8485dc5846ac49368eff9aefb0e22066" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39647 } }
I20260927 18:42:00.338706 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:00.325223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46479 (local address 127.6.4.62:39647)
0927 18:42:00.325634 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:00.325649 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:00.325684 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:42:00.326275 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:00.326290 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:00.326652 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:00.326923 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:00.327744 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:00.328776 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:00.329986 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:00.330620 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:00.330743 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:00.333420 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:00.333426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:00.333439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:00.333468 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:00.335658 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:00.336131 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:00.336137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:00.336143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:00.336253 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:00.336546 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:00.336549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:00.336551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:00.336962 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:00.337155 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:00.337421 (+ 266us) server_negotiation.cc:299] Negotiation successful
0927 18:42:00.337709 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260927 18:42:00.339196 28132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:00.339831 28137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8485dc5846ac49368eff9aefb0e22066. Latest consensus state: current_term: 1 leader_uuid: "8485dc5846ac49368eff9aefb0e22066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8485dc5846ac49368eff9aefb0e22066" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39647 } } }
I20260927 18:42:00.339937 28137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:00.340417 28144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:00.342029 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:42:00.342093 28138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8485dc5846ac49368eff9aefb0e22066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8485dc5846ac49368eff9aefb0e22066" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 39647 } } }
I20260927 18:42:00.342183 28138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:00.343509 28144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:00.348697 28144 catalog_manager.cc:1347] Generated new cluster ID: 8a45575774a14a228011efed8ef73bf7
I20260927 18:42:00.348819 28144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:00.360028 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:00.344332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46895 (local address 127.6.4.62:39647)
0927 18:42:00.344435 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:00.344440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:00.344614 (+ 174us) server_negotiation.cc:407] Connection header received
0927 18:42:00.344726 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:00.344729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:00.344775 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:00.344857 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:00.345624 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:00.346421 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:00.347555 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:00.347903 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:00.348014 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:00.357116 (+ 9102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:00.357120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:00.357122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:00.357154 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:00.358502 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:00.358907 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:00.358910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:00.358911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:00.358983 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:00.359218 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:00.359221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:00.359223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:00.359401 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:00.359517 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:00.359738 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 18:42:00.359873 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:42:00.389454 28144 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:00.390421 28144 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:00.395056 28144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8485dc5846ac49368eff9aefb0e22066: Generated new TSK 0
I20260927 18:42:00.395823 28144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:42:00.409237 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28064
2026-09-27T18:42:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:42:00 dist-test-slave-bzzj krb5kdc[28163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:42:00 dist-test-slave-bzzj krb5kdc[28163](info): set up 2 sockets
Sep 27 18:42:00 dist-test-slave-bzzj krb5kdc[28163](info): commencing operation
krb5kdc: starting...
W20260927 18:42:01.356045 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:42:01 dist-test-slave-bzzj krb5kdc[28163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:42:01.475410 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44917
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:33069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:44917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:01.583962 28176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:01.584287 28176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:01.584352 28176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:01.588433 28176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:01.588514 28176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:01.588541 28176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:01.588562 28176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:01.588582 28176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:01.593986 28176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:33069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:44917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:01.595062 28176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:01.596071 28176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:01.604365 28182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:01.604372 28184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:01.604386 28181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:01.604599 28176 server_base.cc:1034] running on GCE node
I20260927 18:42:01.605268 28176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:01.606359 28176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:01.607532 28176 hybrid_clock.cc:648] HybridClock initialized: now 1790534521607513 us; error 30 us; skew 500 ppm
Sep 27 18:42:01 dist-test-slave-bzzj krb5kdc[28163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:01.610126 28176 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:42:01.611095 28176 webserver.cc:460] Webserver started at http://127.6.4.62:46401/ using document root <none> and password file <none>
I20260927 18:42:01.611649 28176 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:01.611713 28176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:01.611912 28176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:01.614022 28176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "e031a3d3991e4e11b9b2a3c28163f8d3"
format_stamp: "Formatted at 2026-09-27 18:42:01 on dist-test-slave-bzzj"
server_key: "c0d4e7406b5e22dce053372e5f7d3037"
server_key_iv: "b7c10b86755cc5604812546f37596989"
server_key_version: "encryptionkey@0"
I20260927 18:42:01.614555 28176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "e031a3d3991e4e11b9b2a3c28163f8d3"
format_stamp: "Formatted at 2026-09-27 18:42:01 on dist-test-slave-bzzj"
server_key: "c0d4e7406b5e22dce053372e5f7d3037"
server_key_iv: "b7c10b86755cc5604812546f37596989"
server_key_version: "encryptionkey@0"
I20260927 18:42:01.618275 28176 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:42:01.620823 28191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:01.622037 28176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:42:01.622155 28176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "e031a3d3991e4e11b9b2a3c28163f8d3"
format_stamp: "Formatted at 2026-09-27 18:42:01 on dist-test-slave-bzzj"
server_key: "c0d4e7406b5e22dce053372e5f7d3037"
server_key_iv: "b7c10b86755cc5604812546f37596989"
server_key_version: "encryptionkey@0"
I20260927 18:42:01.622256 28176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:01.634084 28176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:01.637267 28176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:01.637471 28176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:01.644058 28176 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:44917
I20260927 18:42:01.644135 28243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:44917 every 8 connection(s)
I20260927 18:42:01.645229 28176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:01.647951 28244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:01.651401 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28176
I20260927 18:42:01.651539 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:42:01.651837 6160 external_mini_cluster.cc:1428] Setting key eafecd6a417408f6ca791d0475571a1d
I20260927 18:42:01.654080 28244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3: Bootstrap starting.
I20260927 18:42:01.656555 28244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:01.657460 28244 log.cc:824] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:01.659487 28244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3: No bootstrap required, opened a new log
I20260927 18:42:01.660853 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:01.653204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58937 (local address 127.6.4.62:44917)
0927 18:42:01.653600 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:01.653615 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:01.653650 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:42:01.654341 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:01.654359 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:01.654724 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:01.655014 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:01.655357 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:01.656370 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:01.657549 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:01.658040 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:01.658183 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:01.658977 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:01.658982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:01.658995 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:01.659371 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:01.659422 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:01.659443 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:01.659530 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:01.659683 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 18:42:01.659900 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":99,"threads_started":1}
I20260927 18:42:01.663481 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:42:01.663365 28244 raft_consensus.cc:348] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44917 } }
I20260927 18:42:01.663640 28244 raft_consensus.cc:374] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:01.663688 28244 raft_consensus.cc:729] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e031a3d3991e4e11b9b2a3c28163f8d3, State: Initialized, Role: FOLLOWER
I20260927 18:42:01.664168 28244 consensus_queue.cc:260] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44917 } }
I20260927 18:42:01.664319 28244 raft_consensus.cc:388] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:01.664390 28244 raft_consensus.cc:482] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:01.664495 28244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:01.665635 28244 raft_consensus.cc:504] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44917 } }
I20260927 18:42:01.665979 28244 leader_election.cc:302] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e031a3d3991e4e11b9b2a3c28163f8d3; no voters:
I20260927 18:42:01.666335 28244 leader_election.cc:288] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:42:01.666412 28254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:01.666589 28254 raft_consensus.cc:686] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [term 1 LEADER]: Becoming Leader. State: Replica: e031a3d3991e4e11b9b2a3c28163f8d3, State: Running, Role: LEADER
I20260927 18:42:01.666894 28254 consensus_queue.cc:237] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44917 } }
I20260927 18:42:01.667639 28244 sys_catalog.cc:564] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:01.668084 28254 sys_catalog.cc:455] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44917 } } }
I20260927 18:42:01.668231 28254 sys_catalog.cc:458] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:01.668416 28254 sys_catalog.cc:455] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e031a3d3991e4e11b9b2a3c28163f8d3. Latest consensus state: current_term: 1 leader_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e031a3d3991e4e11b9b2a3c28163f8d3" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 44917 } } }
I20260927 18:42:01.668499 28254 sys_catalog.cc:458] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:01.669654 28259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:01.670047 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:01.665693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41161 (local address 127.6.4.62:44917)
0927 18:42:01.665849 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:01.665854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:01.666136 (+ 282us) server_negotiation.cc:407] Connection header received
0927 18:42:01.666224 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:01.666226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:01.666267 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:01.666335 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:01.666596 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:01.667184 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:01.668033 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:01.668434 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:01.668512 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:01.669436 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:01.669440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:01.669443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:01.669464 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:01.669492 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:01.669504 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:01.669565 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:01.669885 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 18:42:01.669940 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 18:42:01.672875 28259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:01.679224 28259 catalog_manager.cc:1347] Generated new cluster ID: b7e609687b9d4c44a463b41fc6ec0f4d
I20260927 18:42:01.679289 28259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:01.705118 28259 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:01.706117 28259 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:01.710945 28259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e031a3d3991e4e11b9b2a3c28163f8d3: Generated new TSK 0
I20260927 18:42:01.711624 28259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:42:01.717823 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28176
2026-09-27T18:42:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:42:01 dist-test-slave-bzzj krb5kdc[28275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:42:01 dist-test-slave-bzzj krb5kdc[28275](info): set up 2 sockets
Sep 27 18:42:01 dist-test-slave-bzzj krb5kdc[28275](info): commencing operation
krb5kdc: starting...
W20260927 18:42:02.672039 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:42:02 dist-test-slave-bzzj krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:42:02.788684 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40255
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:34743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:40255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:02.896417 28288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:02.896685 28288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:02.896725 28288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:02.900910 28288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:02.900985 28288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:02.901003 28288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:02.901019 28288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:02.901032 28288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:02.906443 28288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:34743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:40255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:40255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:02.907526 28288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:02.908599 28288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:02.916808 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:02.916975 28288 server_base.cc:1034] running on GCE node
W20260927 18:42:02.916824 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:02.916808 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:02.917641 28288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:02.918784 28288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:02.919948 28288 hybrid_clock.cc:648] HybridClock initialized: now 1790534522919932 us; error 32 us; skew 500 ppm
Sep 27 18:42:02 dist-test-slave-bzzj krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534522, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:02.926599 28288 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:42:02.927578 28288 webserver.cc:460] Webserver started at http://127.6.4.62:35963/ using document root <none> and password file <none>
I20260927 18:42:02.928171 28288 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:02.928239 28288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:02.928432 28288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:02.930526 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "0584546fb6ae4f1c81de067a015788a6"
format_stamp: "Formatted at 2026-09-27 18:42:02 on dist-test-slave-bzzj"
server_key: "eb48a0a7d91a5c6157dbf80c4e9b630b"
server_key_iv: "d3921b107cf5841fba9b96046bdb761a"
server_key_version: "encryptionkey@0"
I20260927 18:42:02.931073 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "0584546fb6ae4f1c81de067a015788a6"
format_stamp: "Formatted at 2026-09-27 18:42:02 on dist-test-slave-bzzj"
server_key: "eb48a0a7d91a5c6157dbf80c4e9b630b"
server_key_iv: "d3921b107cf5841fba9b96046bdb761a"
server_key_version: "encryptionkey@0"
I20260927 18:42:02.934866 28288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 18:42:02.937363 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:02.938601 28288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:42:02.938720 28288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "0584546fb6ae4f1c81de067a015788a6"
format_stamp: "Formatted at 2026-09-27 18:42:02 on dist-test-slave-bzzj"
server_key: "eb48a0a7d91a5c6157dbf80c4e9b630b"
server_key_iv: "d3921b107cf5841fba9b96046bdb761a"
server_key_version: "encryptionkey@0"
I20260927 18:42:02.938823 28288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:02.943390 28288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:02.946374 28288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:02.946552 28288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:02.952877 28288 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:40255
I20260927 18:42:02.953027 28355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:40255 every 8 connection(s)
I20260927 18:42:02.953833 28288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:02.954528 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28288
I20260927 18:42:02.954643 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:42:02.954909 6160 external_mini_cluster.cc:1428] Setting key c1628a8df330764b7df1d22664b14921
I20260927 18:42:02.956995 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:02.960587 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:02.956343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37417 (local address 127.6.4.62:40255)
0927 18:42:02.956752 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:02.956771 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:02.956815 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:42:02.957664 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:02.957687 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:02.958060 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:02.958429 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:02.959067 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:02.959073 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:02.959089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:02.959099 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:02.959157 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:02.959180 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:02.959322 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:02.959450 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:42:02.959604 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260927 18:42:02.963326 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:42:02.963557 28356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6: Bootstrap starting.
I20260927 18:42:02.966006 28356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:02.966820 28356 log.cc:824] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:02.966928 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:02.965308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45903 (local address 127.6.4.62:40255)
0927 18:42:02.965412 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:02.965417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:02.965620 (+ 203us) server_negotiation.cc:407] Connection header received
0927 18:42:02.965733 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:02.965736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:02.965771 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:02.965833 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:02.966529 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:02.966533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:02.966536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:42:02.966548 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:02.966573 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:42:02.966584 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:02.966652 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:02.966723 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 18:42:02.966761 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:42:02.968871 28356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6: No bootstrap required, opened a new log
I20260927 18:42:02.972573 28356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0584546fb6ae4f1c81de067a015788a6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40255 } }
I20260927 18:42:02.972819 28356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:42:02.972869 28356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0584546fb6ae4f1c81de067a015788a6, State: Initialized, Role: FOLLOWER
I20260927 18:42:02.973285 28356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0584546fb6ae4f1c81de067a015788a6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40255 } }
I20260927 18:42:02.973413 28356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:42:02.973476 28356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:42:02.973563 28356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:02.974493 28356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0584546fb6ae4f1c81de067a015788a6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40255 } }
I20260927 18:42:02.974821 28356 leader_election.cc:302] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0584546fb6ae4f1c81de067a015788a6; no voters:
I20260927 18:42:02.975116 28356 leader_election.cc:288] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:42:02.975217 28366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:02.975432 28366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [term 1 LEADER]: Becoming Leader. State: Replica: 0584546fb6ae4f1c81de067a015788a6, State: Running, Role: LEADER
I20260927 18:42:02.975762 28366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0584546fb6ae4f1c81de067a015788a6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40255 } }
I20260927 18:42:02.976329 28356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:02.977505 28367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0584546fb6ae4f1c81de067a015788a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0584546fb6ae4f1c81de067a015788a6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40255 } } }
I20260927 18:42:02.977631 28367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:02.977738 28368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0584546fb6ae4f1c81de067a015788a6. Latest consensus state: current_term: 1 leader_uuid: "0584546fb6ae4f1c81de067a015788a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0584546fb6ae4f1c81de067a015788a6" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 40255 } } }
I20260927 18:42:02.977867 28368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:02.978027 28376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:02.981247 28376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:02.987310 28376 catalog_manager.cc:1347] Generated new cluster ID: 928d1711701a4b8ebb5d102a7127e2e7
I20260927 18:42:02.987383 28376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:03.005884 28376 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:03.006218 28376 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:03.015543 28376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0584546fb6ae4f1c81de067a015788a6: Generated new TSK 0
I20260927 18:42:03.016324 28376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:42:03.021586 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28288
2026-09-27T18:42:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1303 ms)
[----------] 18 tests from AuthTokenIssuingTest (23500 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:42:03 dist-test-slave-bzzj krb5kdc[28387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:42:03 dist-test-slave-bzzj krb5kdc[28387](info): set up 2 sockets
Sep 27 18:42:03 dist-test-slave-bzzj krb5kdc[28387](info): commencing operation
krb5kdc: starting...
W20260927 18:42:03.980058 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:42:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:42:04.095171 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42339
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42339,127.6.4.61:46411,127.6.4.60:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:04.204311 28400 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:42:04.204607 28400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:04.204628 28400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:04.204672 28400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:04.208762 28400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:04.208837 28400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:04.208865 28400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:04.208881 28400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:04.208895 28400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:04.214401 28400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42339,127.6.4.61:46411,127.6.4.60:38907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:42339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:04.215512 28400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:04.216583 28400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:04.224824 28406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:04.224901 28408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:04.224833 28405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:04.225107 28400 server_base.cc:1034] running on GCE node
I20260927 18:42:04.225697 28400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:04.226838 28400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:04.228003 28400 hybrid_clock.cc:648] HybridClock initialized: now 1790534524227986 us; error 34 us; skew 500 ppm
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:04.230850 28400 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:42:04.231734 28400 webserver.cc:460] Webserver started at http://127.6.4.62:33991/ using document root <none> and password file <none>
I20260927 18:42:04.232327 28400 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:04.232389 28400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:04.232596 28400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:04.234652 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "bfa48c2cb71a4450b35572a0db483767"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "8a9a2dad88451be990070b737d5fe072"
server_key_iv: "5b32448ec2e645fe701181e28eb8bd8c"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.235194 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "bfa48c2cb71a4450b35572a0db483767"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "8a9a2dad88451be990070b737d5fe072"
server_key_iv: "5b32448ec2e645fe701181e28eb8bd8c"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.239003 28400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260927 18:42:04.241592 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:04.242839 28400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:42:04.242950 28400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "bfa48c2cb71a4450b35572a0db483767"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "8a9a2dad88451be990070b737d5fe072"
server_key_iv: "5b32448ec2e645fe701181e28eb8bd8c"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.243057 28400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:04.258461 28400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:04.261601 28400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:04.261783 28400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:04.268306 28400 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:42339
I20260927 18:42:04.268560 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:42339 every 8 connection(s)
I20260927 18:42:04.269263 28400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:04.271202 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28400
I20260927 18:42:04.271337 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:42:04.271623 6160 external_mini_cluster.cc:1428] Setting key a0b00787a26f31c3ba2d21595775ca58
I20260927 18:42:04.272321 28468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:04.276371 28468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.4.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.61@KRBTEST.COM" created.
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:04.291616 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.278631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42339 (local address 127.0.0.1:44818)
0927 18:42:04.279169 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.279210 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.280497 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.281263 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.281269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.281606 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.281871 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.281884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.283226 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.283230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.284083 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.284123 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.284866 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.284918 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.285008 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.285605 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.285621 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.287005 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.288785 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.288793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.288804 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.289011 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.289286 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.289289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.289291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.289402 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.289955 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.289961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.290092 (+ 131us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.290237 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.290625 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":116,"spinlock_wait_cycles":57728,"thread_start_us":158,"threads_started":1}
I20260927 18:42:04.291616 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.278631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.6.4.62:42339)
0927 18:42:04.279206 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.279222 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.279316 (+ 94us) server_negotiation.cc:407] Connection header received
0927 18:42:04.280804 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.280809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.281037 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.281206 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.282010 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.283095 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.284257 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.284780 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.284930 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.287138 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.287145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.287157 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.287177 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.288688 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.289106 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.289111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.289112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.289207 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.289477 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.289480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.289482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.289854 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.289971 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.290309 (+ 338us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.290570 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":456,"thread_start_us":158,"threads_started":1}
I20260927 18:42:04.293998 28468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:04.294693 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.294452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52518)
0927 18:42:04.294561 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.294631 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:42:04.294865 28417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab.
Entry for principal kudu/127.6.4.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab.
I20260927 18:42:04.295693 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.295467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52534)
0927 18:42:04.295581 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.295633 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 18:42:04.295811 28468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:46411: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
I20260927 18:42:04.296427 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:46411
--webserver_interface=127.6.4.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab
--principal=kudu/127.6.4.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42339,127.6.4.61:46411,127.6.4.60:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:42:04.345080 28468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } attempt: 1
I20260927 18:42:04.345989 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.345662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52540)
0927 18:42:04.345819 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.345885 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260927 18:42:04.346673 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.346443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52544)
0927 18:42:04.346538 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.346610 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 18:42:04.346851 28468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:46411: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
W20260927 18:42:04.406394 28475 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:42:04.406710 28475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:04.406745 28475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:04.406808 28475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:04.410849 28475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:04.410912 28475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:04.410938 28475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:04.410959 28475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:04.410980 28475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:04.415009 28468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } attempt: 2
I20260927 18:42:04.415848 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.415548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52558)
0927 18:42:04.415701 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.415770 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 18:42:04.416533 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.416326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52562)
0927 18:42:04.416426 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.416476 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:42:04.416652 28468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:46411: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46411: connect: Connection refused (error 111)
I20260927 18:42:04.416637 28475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42339,127.6.4.61:46411,127.6.4.60:38907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab
--principal=kudu/127.6.4.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:46411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.4.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:04.417733 28475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:04.418743 28475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:42:04.427208 28475 server_base.cc:1034] running on GCE node
W20260927 18:42:04.427171 28481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:04.427186 28483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:04.427172 28480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:04.428020 28475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:04.429137 28475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:04.430307 28475 hybrid_clock.cc:648] HybridClock initialized: now 1790534524430286 us; error 31 us; skew 500 ppm
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:04.433125 28475 init.cc:377] Logged in from keytab as kudu/127.6.4.61@KRBTEST.COM (short username kudu)
I20260927 18:42:04.434023 28475 webserver.cc:460] Webserver started at http://127.6.4.61:40869/ using document root <none> and password file <none>
I20260927 18:42:04.434556 28475 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:04.434619 28475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:04.434803 28475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:04.436861 28475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data/instance:
uuid: "35d7f76b09bf4604bb62656cfb0e0800"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "8a8bdb04799ba2b9be98f4efa229a085"
server_key_iv: "ff1503ce1bf8b921065010bd3dad256d"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.437391 28475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal/instance:
uuid: "35d7f76b09bf4604bb62656cfb0e0800"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "8a8bdb04799ba2b9be98f4efa229a085"
server_key_iv: "ff1503ce1bf8b921065010bd3dad256d"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.441054 28475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:42:04.443388 28490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:04.444633 28475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:42:04.444751 28475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal
uuid: "35d7f76b09bf4604bb62656cfb0e0800"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "8a8bdb04799ba2b9be98f4efa229a085"
server_key_iv: "ff1503ce1bf8b921065010bd3dad256d"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.444854 28475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:04.461853 28475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:04.465114 28475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:04.465297 28475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:04.471796 28475 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.61:46411
I20260927 18:42:04.472072 28542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.61:46411 every 8 connection(s)
I20260927 18:42:04.472886 28475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
I20260927 18:42:04.473272 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28475
I20260927 18:42:04.473397 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-1/wal/instance
I20260927 18:42:04.473685 6160 external_mini_cluster.cc:1428] Setting key a0a1f12e53b1889394b2dec588038aaf
I20260927 18:42:04.475868 28543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:04.479568 28543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.4.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.60@KRBTEST.COM" created.
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:04.493335 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.481572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.6.4.62:42339)
0927 18:42:04.481696 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.481700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.482531 (+ 831us) server_negotiation.cc:407] Connection header received
0927 18:42:04.483593 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.483597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.483674 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.483743 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.484953 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.485708 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.486954 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.487334 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.487443 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.489969 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.489972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.489974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.490001 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.491370 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.491852 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.491855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.491856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.491928 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.492295 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.492297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.492298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.492485 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.492577 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.493071 (+ 494us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.493193 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:42:04.494158 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.481903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42339 (local address 127.0.0.1:44824)
0927 18:42:04.482385 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.482429 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.483381 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.483931 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.483938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.484319 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.484810 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.484825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.485885 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.485888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.486819 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.486838 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.487428 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.487476 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.487623 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.488286 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.488303 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.489785 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.491477 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.491483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.491496 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.491748 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.492012 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.492015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.492017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.492218 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.492569 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.492574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.492834 (+ 260us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.492989 (+ 155us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.493252 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":126,"threads_started":1}
I20260927 18:42:04.495532 28543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab.
Entry for principal kudu/127.6.4.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab.
I20260927 18:42:04.498517 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:38907
--webserver_interface=127.6.4.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab
--principal=kudu/127.6.4.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:46709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:42339,127.6.4.61:46411,127.6.4.60:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:42:04.506063 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.496005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52572)
0927 18:42:04.496146 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.496160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.496222 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.496845 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.496848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.497018 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.497094 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.497099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.499609 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.499612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.500261 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.500270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.500815 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.500833 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.500914 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.501337 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.501355 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.502595 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.504660 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.504662 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.504664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.504840 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.505119 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.505121 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.505122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.505184 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.505691 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.505693 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.505766 (+ 73us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.505835 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.505938 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:42:04.506313 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.496126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.6.4.61:46411)
0927 18:42:04.496512 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.496520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.496538 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:42:04.496570 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.496574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.496733 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.496858 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.497188 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.497939 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.500370 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.500729 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.500829 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.502736 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.502747 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.502766 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.502795 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.504554 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.504939 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.504946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.504948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.505040 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.505255 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.505258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.505260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.505601 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.505749 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.505956 (+ 207us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.506115 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":159,"threads_started":1}
I20260927 18:42:04.507812 28468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } attempt: 3
I20260927 18:42:04.507972 28543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:04.508664 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.508393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52580)
0927 18:42:04.508523 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.508599 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260927 18:42:04.508852 28491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
I20260927 18:42:04.509780 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.509519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52592)
0927 18:42:04.509678 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.509721 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":52}
W20260927 18:42:04.509912 28543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:38907: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:42:04.516706 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.508314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52578)
0927 18:42:04.508506 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.508520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.508614 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.508882 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.508885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.509058 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.509171 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.509179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.510119 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.510123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.511506 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.511517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.512013 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.512030 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.512159 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.512637 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.512650 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.513847 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.515307 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.515310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.515312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.515496 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.515729 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.515732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.515734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.515817 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.516195 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.516198 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.516301 (+ 103us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.516388 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.516536 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260927 18:42:04.516706 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.508373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.6.4.61:46411)
0927 18:42:04.508482 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.508486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.508568 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:42:04.508699 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.508702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.508749 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.508826 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.509291 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.510012 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.511649 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.511921 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.512011 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.513974 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.513977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.513979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.514001 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.515204 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.515580 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.515583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.515585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.515652 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.515892 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.515896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.515898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.516067 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.516205 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.516453 (+ 248us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.516558 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:42:04.517719 28468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:04.518395 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.518169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52602)
0927 18:42:04.518290 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.518338 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260927 18:42:04.519032 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.518806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52618)
0927 18:42:04.518944 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.518986 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260927 18:42:04.519134 28468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:38907: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
I20260927 18:42:04.550303 28468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } attempt: 1
I20260927 18:42:04.551111 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.550832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52630)
0927 18:42:04.550980 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.551037 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260927 18:42:04.551849 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.551649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52634)
0927 18:42:04.551747 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.551795 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:42:04.551966 28468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:38907: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
I20260927 18:42:04.559168 28543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } attempt: 1
I20260927 18:42:04.559938 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.559668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52644)
0927 18:42:04.559796 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.559852 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:42:04.560665 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.560490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52654)
0927 18:42:04.560575 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.560617 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:42:04.560772 28543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:38907: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
W20260927 18:42:04.612247 28549 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:42:04.612534 28549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:04.612572 28549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:04.612628 28549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:04.616724 28549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:04.616788 28549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:04.616807 28549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:04.616825 28549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:04.616890 28549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:04.622237 28549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:46709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:42339,127.6.4.61:46411,127.6.4.60:38907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab
--principal=kudu/127.6.4.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:38907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.4.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:04.623376 28549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:04.624471 28549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:42:04.627125 28468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } attempt: 2
I20260927 18:42:04.627897 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.627628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52662)
0927 18:42:04.627763 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.627825 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:42:04.628616 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.628370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52678)
0927 18:42:04.628487 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.628545 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 18:42:04.628834 28468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:38907: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
I20260927 18:42:04.628901 28543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } attempt: 2
I20260927 18:42:04.629524 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.629302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52694)
0927 18:42:04.629420 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.629467 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 18:42:04.630086 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.629908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52698)
0927 18:42:04.630000 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.630041 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 18:42:04.630187 28543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:38907: Network error: Client connection negotiation failed: client connection to 127.6.4.60:38907: connect: Connection refused (error 111)
W20260927 18:42:04.632838 28556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:04.632823 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:04.633029 28549 server_base.cc:1034] running on GCE node
W20260927 18:42:04.632823 28558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:04.633607 28549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:04.634775 28549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:04.635931 28549 hybrid_clock.cc:648] HybridClock initialized: now 1790534524635915 us; error 32 us; skew 500 ppm
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:04.638556 28549 init.cc:377] Logged in from keytab as kudu/127.6.4.60@KRBTEST.COM (short username kudu)
I20260927 18:42:04.639431 28549 webserver.cc:460] Webserver started at http://127.6.4.60:33325/ using document root <none> and password file <none>
I20260927 18:42:04.639968 28549 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:04.640031 28549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:04.640251 28549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:04.642244 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data/instance:
uuid: "6b1039a02e9c4fda9f02ba81534eaad9"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "529c13807045bd839b40fa777fc8870e"
server_key_iv: "f6ec0326cc3c925e0cf79a9ead2b0b37"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.642762 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal/instance:
uuid: "6b1039a02e9c4fda9f02ba81534eaad9"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "529c13807045bd839b40fa777fc8870e"
server_key_iv: "f6ec0326cc3c925e0cf79a9ead2b0b37"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.646389 28549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 18:42:04.648731 28565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:04.650053 28549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:42:04.650159 28549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal
uuid: "6b1039a02e9c4fda9f02ba81534eaad9"
format_stamp: "Formatted at 2026-09-27 18:42:04 on dist-test-slave-bzzj"
server_key: "529c13807045bd839b40fa777fc8870e"
server_key_iv: "f6ec0326cc3c925e0cf79a9ead2b0b37"
server_key_version: "encryptionkey@0"
I20260927 18:42:04.650269 28549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:04.680936 28549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:04.683849 28549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:04.684021 28549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:04.690747 28549 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.60:38907
I20260927 18:42:04.690995 28617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.60:38907 every 8 connection(s)
I20260927 18:42:04.691732 28549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
I20260927 18:42:04.694487 28618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:04.695637 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28549
I20260927 18:42:04.695749 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534257539224-6160-0/minicluster-data/master-2/wal/instance
I20260927 18:42:04.696038 6160 external_mini_cluster.cc:1428] Setting key 78b639aa5a6f97a9b16ad05d55e2ad24
I20260927 18:42:04.698957 28618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:04.709378 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.700871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.6.4.62:42339)
0927 18:42:04.701029 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.701033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.701395 (+ 362us) server_negotiation.cc:407] Connection header received
0927 18:42:04.701506 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.701508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.701553 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.701635 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.702220 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.702737 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.703791 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.704177 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.704273 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.706311 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.706314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.706316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.706342 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.707543 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.707988 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.707991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.707993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.708066 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.708375 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.708377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.708379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.708567 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.708659 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.709158 (+ 499us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.709254 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 18:42:04.710165 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.697394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.6.4.60:38907)
0927 18:42:04.697806 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.697825 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.697879 (+ 54us) server_negotiation.cc:407] Connection header received
0927 18:42:04.698682 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.698705 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.699129 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.699478 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.700149 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.701489 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.702372 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.702853 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.702997 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.705405 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.705412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.705426 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.705456 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.707209 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.707606 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.707612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.707618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.707731 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.707985 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.707987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.707989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.708508 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.708649 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.708888 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.709174 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260927 18:42:04.710165 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.700768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:42339 (local address 127.0.0.1:44834)
0927 18:42:04.701258 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.701304 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.701427 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.701683 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.701687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.702001 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.702112 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.702117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.702843 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.702846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.703681 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.703690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.704267 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.704312 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.704397 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.704835 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.704852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.706171 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.707641 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.707648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.707652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.707897 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.708202 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.708205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.708207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.708284 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.708670 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.708676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.708792 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.708949 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.709236 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":93,"thread_start_us":151,"threads_started":1}
I20260927 18:42:04.711923 28618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:04.719483 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.712379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46411 (local address 127.0.0.1:52590)
0927 18:42:04.712507 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.712520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.712590 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.712800 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.712802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.712954 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.713040 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.713045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.713734 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.713747 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.714244 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.714252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.714635 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.714649 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.714735 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.715178 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.715191 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.716440 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.718138 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.718141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.718142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.718324 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.718593 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.718596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.718597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.718678 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.719047 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.719050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.719146 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.719243 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.719356 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260927 18:42:04.719569 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.712486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.6.4.61:46411)
0927 18:42:04.712617 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.712622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.712637 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:42:04.712677 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.712680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.712717 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.712778 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.713144 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.713641 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.714345 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.714556 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.714638 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.716565 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.716569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.716571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.716595 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.718040 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.718428 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.718432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.718433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.718516 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.718756 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.718758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.718759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.718959 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.719053 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.719299 (+ 246us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.719421 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:42:04.720155 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.712352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.6.4.62:42339)
0927 18:42:04.712458 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.712462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.712669 (+ 207us) server_negotiation.cc:407] Connection header received
0927 18:42:04.712765 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.712768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.712806 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.712898 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.713447 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.713941 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.714659 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.714877 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.714968 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.717116 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.717121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.717124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.717149 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.718460 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.718865 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.718868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.718870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.718963 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.719201 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.719204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.719205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.719408 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.719639 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.719769 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.719867 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:42:04.720633 28618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:04.721321 28543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } attempt: 3
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:42:04.727980 28468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } attempt: 3
I20260927 18:42:04.729388 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.721068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52722)
0927 18:42:04.721209 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.721221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.721293 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.721477 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.721480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.721635 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.721713 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.721720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.722557 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.722560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.723287 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.723296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.723915 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.723928 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.723993 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.724437 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.724450 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.725844 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.727433 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.727436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.727438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.727676 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.727901 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.727904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.727906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.727987 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.728844 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.728846 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.728932 (+ 86us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.729097 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.729218 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260927 18:42:04.729782 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.721118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.6.4.60:38907)
0927 18:42:04.721212 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.721216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.721258 (+ 42us) server_negotiation.cc:407] Connection header received
0927 18:42:04.721369 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.721372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.721407 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.721470 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.721823 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.722461 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.723395 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.723735 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.723825 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.725978 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.725982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.725985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.726011 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.727344 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.727755 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.727758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.727759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.727829 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.728070 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.728074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.728075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.728720 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.728813 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.729557 (+ 744us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.729650 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":29}
I20260927 18:42:04.731068 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.721257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.6.4.61:46411)
0927 18:42:04.721359 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.721363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.721604 (+ 241us) server_negotiation.cc:407] Connection header received
0927 18:42:04.721767 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.721770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.721809 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.721857 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.722610 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.723162 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.724120 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.724462 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.724553 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.727524 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.727528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.727530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.727552 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.729243 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.729667 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.729670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.729672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.729743 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.730038 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.730041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.730042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.730239 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.730447 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.730797 (+ 350us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.730918 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":37,"spinlock_wait_cycles":3200}
I20260927 18:42:04.731297 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.721619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52738)
0927 18:42:04.721817 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.721830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.721886 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.722401 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.722404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.722567 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.722693 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.722699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.723831 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.723834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.724596 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.724604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.725058 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.725073 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.725151 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.725558 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.725572 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.727264 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.729224 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.729227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.729229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.729449 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.730308 (+ 859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.730311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.730313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.730390 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.730895 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.730898 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.730976 (+ 78us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.731049 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.731168 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":38,"spinlock_wait_cycles":896}
I20260927 18:42:04.732909 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.721759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.6.4.60:38907)
0927 18:42:04.722180 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.722188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.722200 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:04.722231 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.722234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.722270 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.722361 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.722794 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.723733 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.724675 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.724943 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.725339 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.727378 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.727382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.727385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.727408 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.729105 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.729786 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.729789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.729791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.730208 (+ 417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.730509 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.730512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.730514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.730799 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.732058 (+ 1259us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.732637 (+ 579us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.732751 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":154,"threads_started":1}
I20260927 18:42:04.733232 28618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: Bootstrap starting.
Sep 27 18:42:04 dist-test-slave-bzzj krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534524, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:42:04.735663 28543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: Bootstrap starting.
I20260927 18:42:04.736164 28618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:04.737057 28618 log.cc:824] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:04.737293 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.728433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:38907 (local address 127.0.0.1:52750)
0927 18:42:04.728874 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:04.728888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:04.728958 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:04.729365 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:04.729369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:04.729553 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:04.729659 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.729666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.730749 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.730752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.731463 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:04.731475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.732051 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.732069 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:04.732187 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.732594 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:04.732606 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:04.733966 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:04.735743 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.735746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.735748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.735993 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.736325 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:04.736328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:04.736330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:04.736425 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:04.736826 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:04.736829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:04.736940 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 18:42:04.737019 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:42:04.737147 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348}
I20260927 18:42:04.737630 28543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:04.737915 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.728609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.6.4.60:38907)
0927 18:42:04.729150 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.729158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.729168 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:42:04.729196 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.729199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.729235 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.729868 (+ 633us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.729985 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.730624 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.731597 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.731939 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.732382 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.734103 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.734106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.734109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.734130 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.735623 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.736136 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.736139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.736141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.736219 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.736529 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.736533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.736534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.736724 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.737560 (+ 836us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.737644 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.737749 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":110,"threads_started":1}
I20260927 18:42:04.738667 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:42:04.738664 28543 log.cc:824] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:04.739305 28618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: No bootstrap required, opened a new log
I20260927 18:42:04.740741 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: Bootstrap starting.
I20260927 18:42:04.739957 28442 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bfa48c2cb71a4450b35572a0db483767"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44844
I20260927 18:42:04.741744 28543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: No bootstrap required, opened a new log
I20260927 18:42:04.743366 28468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:04.743777 28618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
I20260927 18:42:04.744005 28618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1039a02e9c4fda9f02ba81534eaad9, State: Initialized, Role: FOLLOWER
I20260927 18:42:04.744357 28468 log.cc:824] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:04.744359 28618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
I20260927 18:42:04.744926 28627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.745229 28627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.745512 28543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
I20260927 18:42:04.745679 28618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:04.745713 28543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d7f76b09bf4604bb62656cfb0e0800, State: Initialized, Role: FOLLOWER
I20260927 18:42:04.746038 28543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
I20260927 18:42:04.746321 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: No bootstrap required, opened a new log
I20260927 18:42:04.746582 28629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.746820 28629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.747339 28543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:04.749305 28468 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
I20260927 18:42:04.749533 28468 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa48c2cb71a4450b35572a0db483767, State: Initialized, Role: FOLLOWER
I20260927 18:42:04.749822 28468 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
I20260927 18:42:04.750975 28468 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 18:42:04.751099 28640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:42:04.751190 28640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:42:04.752197 28442 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 18:42:04.752310 28442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:04.753587 28442 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
W20260927 18:42:04.754606 28652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260927 18:42:04.751021 28475 master_runner.cc:186] Error getting master registration for 127.6.4.62:42339: OK, instance_id { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" instance_seqno: 1790534524266510 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:42:04.754918 28475 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 18:42:04.755031 28652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:42:04.749388 28549 master_runner.cc:186] Error getting master registration for 127.6.4.62:42339: OK, instance_id { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" instance_seqno: 1790534524266510 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:42:04.755152 28517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfa48c2cb71a4450b35572a0db483767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "35d7f76b09bf4604bb62656cfb0e0800"
I20260927 18:42:04.755235 28549 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:42:04.755378 28517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:04.756537 28517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa48c2cb71a4450b35572a0db483767 in term 1.
I20260927 18:42:04.756985 28592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfa48c2cb71a4450b35572a0db483767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b1039a02e9c4fda9f02ba81534eaad9"
I20260927 18:42:04.757194 28592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:04.758419 28592 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa48c2cb71a4450b35572a0db483767 in term 1.
I20260927 18:42:04.754571 28442 leader_election.cc:288] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [CANDIDATE]: Term 1 election: Requested vote from peers 35d7f76b09bf4604bb62656cfb0e0800 (127.6.4.61:46411), 6b1039a02e9c4fda9f02ba81534eaad9 (127.6.4.60:38907)
W20260927 18:42:04.759213 28665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:42:04.759334 28665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:42:04.754871 28653 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.756951 28417 leader_election.cc:302] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d7f76b09bf4604bb62656cfb0e0800, bfa48c2cb71a4450b35572a0db483767; no voters:
I20260927 18:42:04.760241 28653 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.760339 28653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:04.760406 28653 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [term 1 LEADER]: Becoming Leader. State: Replica: bfa48c2cb71a4450b35572a0db483767, State: Running, Role: LEADER
I20260927 18:42:04.760715 28653 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } }
W20260927 18:42:04.763326 28421 tablet.cc:2369] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:04.776037 28667 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa48c2cb71a4450b35572a0db483767. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.776225 28667 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:04.776657 28676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:04.777582 28676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:04.778918 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.761652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.6.4.62:42339)
0927 18:42:04.761766 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.761771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.763972 (+ 2201us) server_negotiation.cc:407] Connection header received
0927 18:42:04.764212 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.764216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.764266 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.764318 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.765424 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.766209 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.768824 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.769177 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.769251 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.774210 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.774215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.774218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.774246 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.775836 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.777551 (+ 1715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.777554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.777556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.777645 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.777994 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.777997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.777999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.778198 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.778311 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.778631 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.778752 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:42:04.784735 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.763166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.6.4.60:38907)
0927 18:42:04.763274 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.763278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.776377 (+ 13099us) server_negotiation.cc:407] Connection header received
0927 18:42:04.776576 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.776580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.776629 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.776681 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.777449 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.778166 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.779271 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.779647 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.779719 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.781480 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.781484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.781486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.781511 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.782974 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.783415 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.783419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.783420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.783500 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.783766 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.783769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.783771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.783982 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.784080 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.784453 (+ 373us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.784575 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:42:04.787264 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:04.762427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.6.4.61:46411)
0927 18:42:04.762537 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.762541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.767581 (+ 5040us) server_negotiation.cc:407] Connection header received
0927 18:42:04.767753 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.767756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.767809 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.767859 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.769970 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.770718 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.773116 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:04.773461 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:04.773529 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:04.779906 (+ 6377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:04.779921 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:04.779924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:04.779956 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:04.781514 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.785963 (+ 4449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.785968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.785970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.786064 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:04.786343 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:04.786347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:04.786349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:04.786554 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:04.786670 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:04.787000 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 18:42:04.787114 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:42:04.793462 28517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [term 1 FOLLOWER]: Refusing update from remote peer bfa48c2cb71a4450b35572a0db483767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:04.793462 28592 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [term 1 FOLLOWER]: Refusing update from remote peer bfa48c2cb71a4450b35572a0db483767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:04.794179 28667 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:04.794607 28667 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:04.799304 28627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa48c2cb71a4450b35572a0db483767. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.799437 28627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.799757 28629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa48c2cb71a4450b35572a0db483767. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.799880 28629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.807727 28653 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.807902 28653 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:04.808007 28677 mvcc.cc:205] Tried to move back new op lower bound from 7334029413540605952 to 7334029413452107776. Current Snapshot: MvccSnapshot[applied={T|T < 7334029413540605952}]
I20260927 18:42:04.809157 28627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.809295 28627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.812752 28667 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.812885 28667 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:04.813989 28629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa48c2cb71a4450b35572a0db483767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa48c2cb71a4450b35572a0db483767" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 42339 } } peers { permanent_uuid: "35d7f76b09bf4604bb62656cfb0e0800" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46411 } } peers { permanent_uuid: "6b1039a02e9c4fda9f02ba81534eaad9" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 38907 } } }
I20260927 18:42:04.814113 28629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:04.815042 28679 mvcc.cc:205] Tried to move back new op lower bound from 7334029413540605952 to 7334029413452107776. Current Snapshot: MvccSnapshot[applied={T|T < 7334029413540605952}]
I20260927 18:42:04.819063 28680 mvcc.cc:205] Tried to move back new op lower bound from 7334029413540605952 to 7334029413452107776. Current Snapshot: MvccSnapshot[applied={T|T < 7334029413540605952}]
I20260927 18:42:04.820550 28676 catalog_manager.cc:1347] Generated new cluster ID: 845638bb5f2541d1be48fad416eeb311
I20260927 18:42:04.820621 28676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:04.857201 28676 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:04.858542 28676 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:04.877420 28676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfa48c2cb71a4450b35572a0db483767: Generated new TSK 0
I20260927 18:42:04.878371 28676 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:42:04.908352 28471 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.905573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.6.4.62:42339)
0927 18:42:04.905713 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.905718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.907132 (+ 1414us) server_negotiation.cc:407] Connection header received
0927 18:42:04.907351 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.907355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.907409 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.907465 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.908195 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44856: BlockingRecv error: recv got EOF from 127.0.0.1:44856 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260927 18:42:04.909861 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.906479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.6.4.61:46411)
0927 18:42:04.906600 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.906604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.908907 (+ 2303us) server_negotiation.cc:407] Connection header received
0927 18:42:04.909092 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.909095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.909148 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.909199 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.909722 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52622: BlockingRecv error: recv got EOF from 127.0.0.1:52622 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260927 18:42:04.911656 28624 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.910345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.6.4.60:38907)
0927 18:42:04.910478 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.910483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.910851 (+ 368us) server_negotiation.cc:407] Connection header received
0927 18:42:04.910989 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.910992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.911039 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.911135 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.911537 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52778: BlockingRecv error: recv got EOF from 127.0.0.1:52778 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260927 18:42:04.918351 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.916592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.6.4.61:46411)
0927 18:42:04.916715 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.916720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.917263 (+ 543us) server_negotiation.cc:407] Connection header received
0927 18:42:04.917508 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.917512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.917592 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.917653 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.918228 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52630: BlockingRecv error: recv got EOF from 127.0.0.1:52630 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260927 18:42:04.923655 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.922113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.6.4.61:46411)
0927 18:42:04.922212 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.922216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.922703 (+ 487us) server_negotiation.cc:407] Connection header received
0927 18:42:04.922904 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.922908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.922961 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.923016 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.923542 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52632: BlockingRecv error: recv got EOF from 127.0.0.1:52632 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 18:42:04.929401 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.927871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.6.4.61:46411)
0927 18:42:04.927970 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.927974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.928527 (+ 553us) server_negotiation.cc:407] Connection header received
0927 18:42:04.928695 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.928698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.928745 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.928790 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.929292 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52648: BlockingRecv error: recv got EOF from 127.0.0.1:52648 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 18:42:04.938185 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.936523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.6.4.61:46411)
0927 18:42:04.936667 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.936673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.937215 (+ 542us) server_negotiation.cc:407] Connection header received
0927 18:42:04.937418 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.937421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.937475 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.937525 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.938059 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52656: BlockingRecv error: recv got EOF from 127.0.0.1:52656 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260927 18:42:04.950381 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.949094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.6.4.61:46411)
0927 18:42:04.949206 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.949211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.949592 (+ 381us) server_negotiation.cc:407] Connection header received
0927 18:42:04.949725 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.949729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.949782 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.949862 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.950264 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52672: BlockingRecv error: recv got EOF from 127.0.0.1:52672 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260927 18:42:04.969601 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:04.968568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.6.4.61:46411)
0927 18:42:04.968697 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:04.968701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:04.968822 (+ 121us) server_negotiation.cc:407] Connection header received
0927 18:42:04.968946 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:04.968950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:04.969006 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:04.969105 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:04.969483 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52676: BlockingRecv error: recv got EOF from 127.0.0.1:52676 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260927 18:42:05.004523 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:05.003484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.6.4.61:46411)
0927 18:42:05.003626 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.003629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.003788 (+ 159us) server_negotiation.cc:407] Connection header received
0927 18:42:05.003871 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.003874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.003932 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.004011 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:05.004427 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52684: BlockingRecv error: recv got EOF from 127.0.0.1:52684 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260927 18:42:05.071491 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:05.070587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.6.4.61:46411)
0927 18:42:05.070711 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.070714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.070820 (+ 106us) server_negotiation.cc:407] Connection header received
0927 18:42:05.070944 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.070946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.070991 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.071067 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:05.071394 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52686: BlockingRecv error: recv got EOF from 127.0.0.1:52686 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 18:42:05.202652 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:05.201626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.6.4.61:46411)
0927 18:42:05.201796 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.201800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.201911 (+ 111us) server_negotiation.cc:407] Connection header received
0927 18:42:05.202033 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.202035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.202082 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.202151 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:05.202531 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52702: BlockingRecv error: recv got EOF from 127.0.0.1:52702 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260927 18:42:05.462705 28548 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:05.461361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52706 (local address 127.6.4.61:46411)
0927 18:42:05.461540 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.461547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.461713 (+ 166us) server_negotiation.cc:407] Connection header received
0927 18:42:05.461892 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.461896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.461958 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.462037 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:05.462583 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52706: BlockingRecv error: recv got EOF from 127.0.0.1:52706 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 18:42:05.752578 28640 catalog_manager.cc:1259] Loaded cluster ID: 845638bb5f2541d1be48fad416eeb311
I20260927 18:42:05.752657 28640 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: loading cluster ID for follower catalog manager: success
I20260927 18:42:05.754396 28640 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: acquiring CA information for follower catalog manager: success
I20260927 18:42:05.755594 28640 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b1039a02e9c4fda9f02ba81534eaad9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:42:05.756232 28652 catalog_manager.cc:1259] Loaded cluster ID: 845638bb5f2541d1be48fad416eeb311
I20260927 18:42:05.756291 28652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: loading cluster ID for follower catalog manager: success
I20260927 18:42:05.757782 28652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: acquiring CA information for follower catalog manager: success
I20260927 18:42:05.758790 28652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 35d7f76b09bf4604bb62656cfb0e0800: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:42:05.980814 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:05.976847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52708 (local address 127.6.4.61:46411)
0927 18:42:05.977084 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.977087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.977104 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:42:05.977226 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.977229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.977362 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.977485 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:05.977817 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.978397 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.979272 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.979506 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.979583 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:05.979904 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:05.980451 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:05.980544 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:05.980607 (+ 63us) server_negotiation.cc:299] Negotiation successful
0927 18:42:05.980660 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260927 18:42:05.987218 28755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:05.983823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.6.4.60:38907)
0927 18:42:05.984027 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.984031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.984043 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:05.984153 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.984156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.984279 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.984378 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:05.984548 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.985060 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.985850 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.986083 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.986166 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:05.986427 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:05.986844 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:05.986932 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:05.987012 (+ 80us) server_negotiation.cc:299] Negotiation successful
0927 18:42:05.987062 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":60,"threads_started":1}
I20260927 18:42:05.993825 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:05.990322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.6.4.61:46411)
0927 18:42:05.990445 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.990449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.990464 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:42:05.990679 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.990683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.990732 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.990795 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:05.991000 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.991786 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.992760 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.992975 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.993057 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:05.993384 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:05.993468 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:05.993527 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:05.993621 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 18:42:05.993671 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:42:05.994205 28755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:05.990190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.6.4.60:38907)
0927 18:42:05.990300 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.990303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.990601 (+ 298us) server_negotiation.cc:407] Connection header received
0927 18:42:05.990756 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.990760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.990799 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.990855 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:05.991136 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.991871 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.993055 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.993350 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.993435 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:05.993738 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:05.993836 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:05.993880 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:05.994033 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 18:42:05.994082 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:42:05.994570 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:05.990150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.6.4.62:42339)
0927 18:42:05.990409 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:05.990414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:05.990577 (+ 163us) server_negotiation.cc:407] Connection header received
0927 18:42:05.990965 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:05.990968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:05.991125 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:05.991238 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:05.991470 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.992053 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.993098 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:05.993320 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:05.993417 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:05.993767 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:05.994114 (+ 347us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:05.994185 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:05.994380 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 18:42:05.994441 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260927 18:42:05.997364 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28400
I20260927 18:42:06.003000 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28475
I20260927 18:42:06.007836 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28549
2026-09-27T18:42:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2987 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:42:06 dist-test-slave-bzzj krb5kdc[28766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:42:06 dist-test-slave-bzzj krb5kdc[28766](info): set up 2 sockets
Sep 27 18:42:06 dist-test-slave-bzzj krb5kdc[28766](info): commencing operation
krb5kdc: starting...
W20260927 18:42:06.960115 6160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/test-user.keytab.
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:42:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:42:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.4.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
Entry for principal kudu/127.6.4.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab.
I20260927 18:42:07.076864 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:43185
--webserver_interface=127.6.4.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:43185,127.6.4.61:46631,127.6.4.60:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:42:07.186252 28779 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:42:07.186558 28779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:07.186580 28779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:07.186617 28779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:07.190577 28779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:07.190642 28779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:07.190660 28779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:07.190676 28779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:07.190689 28779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:07.196081 28779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:43185,127.6.4.61:46631,127.6.4.60:36481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.62.keytab
--principal=kudu/127.6.4.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.62:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.4.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:07.197189 28779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:07.198160 28779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:07.206341 28785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:07.206363 28787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:07.206605 28779 server_base.cc:1034] running on GCE node
W20260927 18:42:07.206368 28784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:07.208139 28779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:07.209261 28779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:07.210424 28779 hybrid_clock.cc:648] HybridClock initialized: now 1790534527210397 us; error 39 us; skew 500 ppm
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:07.213143 28779 init.cc:377] Logged in from keytab as kudu/127.6.4.62@KRBTEST.COM (short username kudu)
I20260927 18:42:07.214035 28779 webserver.cc:460] Webserver started at http://127.6.4.62:43181/ using document root <none> and password file <none>
I20260927 18:42:07.214593 28779 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:07.214649 28779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:07.214848 28779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:07.216907 28779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data/instance:
uuid: "290b182021f44713bf2daaf5bedd1e43"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "ef466fbac6c0e733f80d8dde84945b56"
server_key_iv: "ce954c0976d05cd40841c0ad2c382d41"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.217433 28779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal/instance:
uuid: "290b182021f44713bf2daaf5bedd1e43"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "ef466fbac6c0e733f80d8dde84945b56"
server_key_iv: "ce954c0976d05cd40841c0ad2c382d41"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.221107 28779 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:42:07.223431 28794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:07.224743 28779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:42:07.224859 28779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal
uuid: "290b182021f44713bf2daaf5bedd1e43"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "ef466fbac6c0e733f80d8dde84945b56"
server_key_iv: "ce954c0976d05cd40841c0ad2c382d41"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.224969 28779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:07.239193 28779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:07.242457 28779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:07.242645 28779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:07.249267 28779 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.62:43185
I20260927 18:42:07.249501 28846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.62:43185 every 8 connection(s)
I20260927 18:42:07.250236 28779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/data/info.pb
I20260927 18:42:07.253042 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28779
I20260927 18:42:07.253171 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-0/wal/instance
I20260927 18:42:07.253207 28847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:07.253513 6160 external_mini_cluster.cc:1428] Setting key c56c4590eceacd19d227a7f4aebe717c
I20260927 18:42:07.257603 28847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.4.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.61@KRBTEST.COM" created.
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:07.273074 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.260200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43185 (local address 127.0.0.1:33790)
0927 18:42:07.260760 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.260807 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.261754 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.262432 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.262437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.262862 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.263104 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.263117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.264587 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.264591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.265340 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.265349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.266141 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.266192 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.266274 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.266818 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.266834 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.268308 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.270102 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.270110 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.270122 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.270339 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.270629 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.270632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.270634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.270739 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.271339 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.271345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.271490 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.271640 (+ 150us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.272011 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":67,"thread_start_us":166,"threads_started":1}
I20260927 18:42:07.273074 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.261227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.6.4.62:43185)
0927 18:42:07.261529 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.261540 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.261587 (+ 47us) server_negotiation.cc:407] Connection header received
0927 18:42:07.262048 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.262052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.262256 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.262421 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.263244 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.264460 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.265468 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.266041 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.266208 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.268447 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.268452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.268464 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.268484 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.269986 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.270439 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.270444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.270445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.270548 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.270828 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.270831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.270833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.271216 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.271357 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.271715 (+ 358us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.271953 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260927 18:42:07.275602 28847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:07.276423 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.276082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37192)
0927 18:42:07.276242 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.276316 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260927 18:42:07.276645 28796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
I20260927 18:42:07.277446 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.277220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37198)
0927 18:42:07.277351 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.277394 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260927 18:42:07.277570 28847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:46631: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab.
Entry for principal kudu/127.6.4.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab.
I20260927 18:42:07.278594 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:46631
--webserver_interface=127.6.4.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab
--principal=kudu/127.6.4.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:43185,127.6.4.61:46631,127.6.4.60:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:42:07.326877 28847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } attempt: 1
I20260927 18:42:07.327798 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.327492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37204)
0927 18:42:07.327633 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.327693 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 18:42:07.328626 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.328346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37214)
0927 18:42:07.328513 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.328570 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260927 18:42:07.328751 28847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:46631: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
W20260927 18:42:07.389395 28854 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:42:07.389701 28854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:07.389736 28854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:07.389799 28854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:07.393929 28854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:07.393997 28854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:07.394022 28854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:07.394044 28854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:07.394073 28854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:07.396925 28847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } attempt: 2
I20260927 18:42:07.397738 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.397443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37226)
0927 18:42:07.397598 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.397662 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 18:42:07.398375 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.398180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37240)
0927 18:42:07.398275 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.398320 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260927 18:42:07.398479 28847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.61:46631: Network error: Client connection negotiation failed: client connection to 127.6.4.61:46631: connect: Connection refused (error 111)
I20260927 18:42:07.399711 28854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:43185,127.6.4.61:46631,127.6.4.60:36481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.61.keytab
--principal=kudu/127.6.4.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.61:46631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.4.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:07.400823 28854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:07.401836 28854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:42:07.410161 28859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:07.410225 28854 server_base.cc:1034] running on GCE node
W20260927 18:42:07.410161 28860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:07.410161 28862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:07.410952 28854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:07.412231 28854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:07.413400 28854 hybrid_clock.cc:648] HybridClock initialized: now 1790534527413367 us; error 47 us; skew 500 ppm
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:07.416028 28854 init.cc:377] Logged in from keytab as kudu/127.6.4.61@KRBTEST.COM (short username kudu)
I20260927 18:42:07.416950 28854 webserver.cc:460] Webserver started at http://127.6.4.61:41693/ using document root <none> and password file <none>
I20260927 18:42:07.417491 28854 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:07.417555 28854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:07.417742 28854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:07.419788 28854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data/instance:
uuid: "17b6d21913a74343984bf9e939b3d547"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "47d77758af6102c7874b828afd2b9173"
server_key_iv: "b0df5af68cb894645b849d5bbb4fdc62"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.420342 28854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal/instance:
uuid: "17b6d21913a74343984bf9e939b3d547"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "47d77758af6102c7874b828afd2b9173"
server_key_iv: "b0df5af68cb894645b849d5bbb4fdc62"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.424052 28854 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260927 18:42:07.426457 28869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:07.427735 28854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:42:07.427847 28854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal
uuid: "17b6d21913a74343984bf9e939b3d547"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "47d77758af6102c7874b828afd2b9173"
server_key_iv: "b0df5af68cb894645b849d5bbb4fdc62"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.427964 28854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:07.442246 28854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:07.445274 28854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:07.445463 28854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:07.452391 28854 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.61:46631
I20260927 18:42:07.452644 28921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.61:46631 every 8 connection(s)
I20260927 18:42:07.453380 28854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/data/info.pb
I20260927 18:42:07.454955 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28854
I20260927 18:42:07.455058 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-1/wal/instance
I20260927 18:42:07.455333 6160 external_mini_cluster.cc:1428] Setting key 6dfd5d72854b28edad61a8a0d701bb59
I20260927 18:42:07.456264 28922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:07.459952 28922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.4.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.4.60@KRBTEST.COM" created.
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:07.474116 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.461919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.6.4.62:43185)
0927 18:42:07.462062 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.462066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.462934 (+ 868us) server_negotiation.cc:407] Connection header received
0927 18:42:07.463958 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.463962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.464025 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.464117 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.465547 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.466073 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.467531 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.467857 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.467966 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.470644 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.470648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.470651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.470684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.472212 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.472691 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.472695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.472697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.472781 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.473067 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.473069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.473070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.473249 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.473320 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.473818 (+ 498us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.473960 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:42:07.474973 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.462251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43185 (local address 127.0.0.1:33792)
0927 18:42:07.462719 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.462763 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.463748 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.464302 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.464308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.464734 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.465390 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.465409 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.466206 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.466209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.467414 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.467425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.467949 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.467999 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.468184 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.468793 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.468809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.470452 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.472319 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.472324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.472338 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.472596 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.472862 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.472864 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.472865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.472995 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.473320 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.473325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.473584 (+ 259us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.473761 (+ 177us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.474073 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260927 18:42:07.476459 28922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.4.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab.
Entry for principal kudu/127.6.4.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab.
I20260927 18:42:07.480486 6160 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
/tmp/dist-test-taskFEZSEE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:36481
--webserver_interface=127.6.4.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab
--principal=kudu/127.6.4.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.4.20:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.4.62:43185,127.6.4.61:46631,127.6.4.60:36481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:42:07.485430 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.476918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37250)
0927 18:42:07.477042 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.477056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.477131 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.477731 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.477734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.477862 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.477983 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.477989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.478946 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.478949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.479634 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.479645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.480390 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.480407 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.480486 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.480900 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.480911 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.482044 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.483920 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.483923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.483925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.484173 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.484440 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.484442 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.484443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.484511 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.485030 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.485033 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.485112 (+ 79us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.485191 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.485297 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 18:42:07.485651 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.477072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.6.4.61:46631)
0927 18:42:07.477394 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.477405 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.477428 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:42:07.477464 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.477468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.477621 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.477723 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.478075 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.478841 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.479734 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.480295 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.480409 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.482167 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.482177 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.482190 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.482210 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.483811 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.484269 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.484275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.484276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.484363 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.484586 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.484589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.484591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.484933 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.485077 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.485304 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.485461 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":154,"threads_started":1}
I20260927 18:42:07.487504 28922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:07.488205 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.487942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36266)
0927 18:42:07.488046 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.488148 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:42:07.488394 28873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
I20260927 18:42:07.489243 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.489061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36272)
0927 18:42:07.489154 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.489195 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260927 18:42:07.489379 28922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:36481: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
I20260927 18:42:07.489624 28847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } attempt: 3
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:42:07.498052 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.490107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.6.4.61:46631)
0927 18:42:07.490202 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.490206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.490222 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:42:07.490315 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.490318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.490366 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.490449 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.490806 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.491593 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.492751 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.493131 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.493220 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.495317 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.495319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.495321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.495343 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.496604 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.497035 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.497038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.497039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.497103 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.497353 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.497355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.497356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.497516 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.497582 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.497836 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.497935 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:42:07.498095 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.490053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37260)
0927 18:42:07.490170 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.490184 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.490252 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.490458 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.490459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.490601 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.490702 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.490706 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.491687 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.491689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.492653 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.492663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.493220 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.493236 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.493312 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.493768 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.493779 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.495189 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.496695 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.496699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.496701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.496949 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.497180 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.497183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.497185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.497277 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.497589 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.497592 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.497693 (+ 101us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.497792 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.497924 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 18:42:07.499198 28847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:42:07.499871 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.499615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36282)
0927 18:42:07.499735 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.499802 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:42:07.500553 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.500320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36298)
0927 18:42:07.500453 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.500499 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260927 18:42:07.500661 28847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:36481: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
I20260927 18:42:07.531800 28847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } attempt: 1
I20260927 18:42:07.532730 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.532376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36302)
0927 18:42:07.532563 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.532640 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260927 18:42:07.533476 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.533260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36304)
0927 18:42:07.533363 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.533414 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 18:42:07.533625 28847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:36481: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
I20260927 18:42:07.538615 28922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } attempt: 1
I20260927 18:42:07.539376 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.539093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36306)
0927 18:42:07.539245 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.539307 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260927 18:42:07.540056 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.539862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36314)
0927 18:42:07.539962 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.540005 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 18:42:07.540179 28922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:36481: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
W20260927 18:42:07.594542 28928 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:42:07.594854 28928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:42:07.594887 28928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:42:07.594949 28928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:42:07.599264 28928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:42:07.599346 28928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:42:07.599375 28928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:42:07.599395 28928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:42:07.599417 28928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:42:07.604936 28928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.4.20:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.4.62:43185,127.6.4.61:46631,127.6.4.60:36481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/krb5kdc/kudu_127.6.4.60.keytab
--principal=kudu/127.6.4.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.4.60:36481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.4.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 27 Sep 2026 17:58:57 UTC on ccycloud.quasar-mffepp.root.comops.site
build id 4676274
I20260927 18:42:07.606038 28928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:42:07.607198 28928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:42:07.608315 28922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } attempt: 2
I20260927 18:42:07.608789 28847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } attempt: 2
I20260927 18:42:07.609134 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.608819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36318)
0927 18:42:07.608975 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.609047 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 18:42:07.609452 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.609149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36326)
0927 18:42:07.609323 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.609380 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260927 18:42:07.609869 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.609649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36340)
0927 18:42:07.609758 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.609808 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:42:07.610005 28922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:36481: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
I20260927 18:42:07.610278 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.610105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36356)
0927 18:42:07.610189 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.610230 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260927 18:42:07.610402 28847 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.4.60:36481: Network error: Client connection negotiation failed: client connection to 127.6.4.60:36481: connect: Connection refused (error 111)
W20260927 18:42:07.615856 28937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:42:07.615890 28935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:07.615989 28928 server_base.cc:1034] running on GCE node
W20260927 18:42:07.615898 28934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:42:07.616701 28928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:42:07.617810 28928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:42:07.618960 28928 hybrid_clock.cc:648] HybridClock initialized: now 1790534527618942 us; error 31 us; skew 500 ppm
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:42:07.621579 28928 init.cc:377] Logged in from keytab as kudu/127.6.4.60@KRBTEST.COM (short username kudu)
I20260927 18:42:07.622516 28928 webserver.cc:460] Webserver started at http://127.6.4.60:37349/ using document root <none> and password file <none>
I20260927 18:42:07.623029 28928 fs_manager.cc:359] Metadata directory not provided
I20260927 18:42:07.623085 28928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:42:07.623253 28928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:42:07.625269 28928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data/instance:
uuid: "3c36c09af0ab40e28c7837b6626c8014"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "e01fe467fd73e36550055e3043c8c09e"
server_key_iv: "7a928642dcdc77d6738bd31df1f951ca"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.625777 28928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal/instance:
uuid: "3c36c09af0ab40e28c7837b6626c8014"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "e01fe467fd73e36550055e3043c8c09e"
server_key_iv: "7a928642dcdc77d6738bd31df1f951ca"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.629406 28928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260927 18:42:07.631759 28944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:42:07.632989 28928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 18:42:07.633098 28928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data,/tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal
uuid: "3c36c09af0ab40e28c7837b6626c8014"
format_stamp: "Formatted at 2026-09-27 18:42:07 on dist-test-slave-bzzj"
server_key: "e01fe467fd73e36550055e3043c8c09e"
server_key_iv: "7a928642dcdc77d6738bd31df1f951ca"
server_key_version: "encryptionkey@0"
I20260927 18:42:07.633208 28928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:42:07.645766 28928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:42:07.648474 28928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:42:07.648660 28928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:42:07.655185 28928 rpc_server.cc:307] RPC server started. Bound to: 127.6.4.60:36481
I20260927 18:42:07.655424 28996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.4.60:36481 every 8 connection(s)
I20260927 18:42:07.656203 28928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/data/info.pb
I20260927 18:42:07.657291 6160 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu as pid 28928
I20260927 18:42:07.657428 6160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFEZSEE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534257539224-6160-0/minicluster-data/master-2/wal/instance
I20260927 18:42:07.657755 6160 external_mini_cluster.cc:1428] Setting key ca35ce4dd759c94f7a2f741a69e2eab4
I20260927 18:42:07.659320 28997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:42:07.663378 28997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
I20260927 18:42:07.671649 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.659029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.6.4.60:36481)
0927 18:42:07.659419 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.659439 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.659482 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:42:07.660179 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.660197 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.660544 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.660824 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.661666 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.662661 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.663795 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.664386 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.664503 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.667055 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.667059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.667068 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.667093 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.668646 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.669108 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.669113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.669117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.669228 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.669496 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.669499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.669501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.670023 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.670219 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.670477 (+ 258us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.670741 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260927 18:42:07.673555 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.665182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.62:43185 (local address 127.0.0.1:33798)
0927 18:42:07.665563 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.665601 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.665712 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.666010 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.666017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.666343 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.666463 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.666468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.667340 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.667342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.667949 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.667957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.668392 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.668407 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.668506 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.668935 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.668949 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.670510 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.672003 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.672010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.672016 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.672295 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.672523 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.672526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.672528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.672618 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.673031 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.673036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.673129 (+ 93us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.673237 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.673366 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"spinlock_wait_cycles":2816,"thread_start_us":141,"threads_started":1}
I20260927 18:42:07.673557 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.665238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33798 (local address 127.6.4.62:43185)
0927 18:42:07.665355 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.665359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.665677 (+ 318us) server_negotiation.cc:407] Connection header received
0927 18:42:07.665807 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.665811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.665856 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.665919 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.666595 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.667231 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.668056 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.668297 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.668385 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.670680 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.670683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.670685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.670707 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.671916 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.672379 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.672381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.672382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.672446 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.672697 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.672699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.672701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.672937 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.673029 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.673314 (+ 285us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.673431 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:42:07.674554 28997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.62@KRBTEST.COM
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:42:07.682225 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.674049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.6.4.62:43185)
0927 18:42:07.674159 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.674163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.674282 (+ 119us) server_negotiation.cc:407] Connection header received
0927 18:42:07.674397 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.674399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.674442 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.674551 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.675213 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.675803 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.676728 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.676930 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.676999 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.679198 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.679202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.679204 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.679230 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.680644 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.681019 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.681022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.681023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.681092 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.681362 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.681364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.681365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.681576 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.681789 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.681915 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.682060 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:42:07.682695 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.675107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.6.4.61:46631)
0927 18:42:07.675215 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.675218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.675230 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:07.675291 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.675293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.675346 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.675425 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.675721 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.676295 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.677143 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.677347 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.677419 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.679591 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.679595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.679598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.679625 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.680914 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.681500 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.681503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.681505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.681584 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.681833 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.681836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.681837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.682044 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.682134 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.682453 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.682554 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:42:07.682695 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.675001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.61:46631 (local address 127.0.0.1:37276)
0927 18:42:07.675146 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.675157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.675216 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.675426 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.675428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.675552 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.675633 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.675638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.676393 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.676397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.677036 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.677046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.677436 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.677451 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.677549 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.677898 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.677911 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.679461 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.681188 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.681191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.681193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.681406 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.681663 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.681666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.681667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.681747 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.682178 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.682180 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.682269 (+ 89us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.682397 (+ 128us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.682540 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260927 18:42:07.683439 28997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.60@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.4.61@KRBTEST.COM
I20260927 18:42:07.691371 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.683864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36364)
0927 18:42:07.684006 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.684018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.684118 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.684297 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.684300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.684439 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.684509 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.684515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.685188 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.685191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.685679 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.685686 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.686092 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.686107 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.686201 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.686667 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.686680 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.687801 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.689987 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.689990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.689992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.690168 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.690408 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.690411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.690414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.690499 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.690887 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.690890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.690972 (+ 82us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.691073 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.691191 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":16}
I20260927 18:42:07.691406 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.683924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36364 (local address 127.6.4.60:36481)
0927 18:42:07.684012 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.684016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.684055 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:42:07.684182 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.684185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.684219 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.684279 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.684602 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.685100 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.685773 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.686016 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.686096 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.687917 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.687921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.687923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.687942 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.689885 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.690242 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.690244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.690245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.690310 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.690579 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.690582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.690583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.690802 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.690878 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.691142 (+ 264us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.691245 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":32}
I20260927 18:42:07.692014 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.683604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.6.4.61:46631)
0927 18:42:07.683698 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.683702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.683840 (+ 138us) server_negotiation.cc:407] Connection header received
0927 18:42:07.683949 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.683952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.683995 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.684055 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.684627 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.685099 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.685781 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.685997 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.686077 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.688520 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.688524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.688526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.688552 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.689887 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.690582 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.690585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.690587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.690671 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.690900 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.690904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.690905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.691068 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.691294 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.691408 (+ 114us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.691877 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:42:07.694407 28997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: Bootstrap starting.
I20260927 18:42:07.694936 6160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:42:07.696743 28997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:07.696048 28821 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "290b182021f44713bf2daaf5bedd1e43"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33806
I20260927 18:42:07.697472 28997 log.cc:824] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:07.699330 28997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: No bootstrap required, opened a new log
I20260927 18:42:07.701160 28922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } attempt: 3
I20260927 18:42:07.702679 28997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.702939 28997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c36c09af0ab40e28c7837b6626c8014, State: Initialized, Role: FOLLOWER
I20260927 18:42:07.703286 28997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.703843 29003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.704042 29003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.704520 28997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: configured and running, proceeding with master startup.
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.61@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:42:07.709539 28847 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } attempt: 3
W20260927 18:42:07.709583 29014 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:42:07.709649 29014 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:42:07.710218 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.701608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36380)
0927 18:42:07.701741 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.701756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.701848 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.702081 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.702084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.702280 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.702386 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.702392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.703310 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.703314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.704076 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.704088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.704643 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.704661 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.704733 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.705239 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.705254 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.706652 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.708622 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.708625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.708627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.708864 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.709171 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.709174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.709176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.709261 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.709695 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.709698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.709800 (+ 102us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.709896 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.710051 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:42:07.710649 29000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.701684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.6.4.60:36481)
0927 18:42:07.701818 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.701823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.701839 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:42:07.701936 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.701939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.701984 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.702068 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.702503 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.703202 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.704227 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.704561 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.704648 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.706801 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.706805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.706808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.706840 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.708498 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.708969 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.708972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.708974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.709074 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.709353 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.709357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.709358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.709561 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.709650 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.710408 (+ 758us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.710514 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:42:07.708140 28928 master_runner.cc:186] Error getting master registration for 127.6.4.62:43185: OK, instance_id { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" instance_seqno: 1790534527247431 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:42:07.711819 28928 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:42:07.713596 28922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: Bootstrap starting.
I20260927 18:42:07.716252 28922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: Neither blocks nor log segments found. Creating new log.
Sep 27 18:42:07 dist-test-slave-bzzj krb5kdc[28766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.4.62@KRBTEST.COM for kudu/127.6.4.60@KRBTEST.COM
I20260927 18:42:07.717113 28922 log.cc:824] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:07.719497 28922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: No bootstrap required, opened a new log
I20260927 18:42:07.720873 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.709943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.4.60:36481 (local address 127.0.0.1:36388)
0927 18:42:07.710104 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:42:07.710116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:42:07.710186 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:42:07.711422 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:42:07.711425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:42:07.711619 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:42:07.711730 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.711738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.712971 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.712975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.713846 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:42:07.713858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.714496 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.714515 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:42:07.714613 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.715121 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:42:07.715136 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:42:07.716848 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:42:07.719383 (+ 2535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.719386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.719389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.719603 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.719845 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:42:07.719849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:42:07.719851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:42:07.719930 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:42:07.720355 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:42:07.720358 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:42:07.720450 (+ 92us) client_negotiation.cc:769] Sending connection context
0927 18:42:07.720549 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:42:07.720696 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260927 18:42:07.720901 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.710021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36388 (local address 127.6.4.60:36481)
0927 18:42:07.711066 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.711079 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.711091 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:42:07.711151 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.711153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.711260 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.711977 (+ 717us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.712065 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.712861 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.713975 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.714355 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.714499 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.717019 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.717024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.717027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.717077 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.719281 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.719682 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.719686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.719688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.719774 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.720018 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.720021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.720023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.720263 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.720345 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.720615 (+ 270us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.720724 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":98,"threads_started":1}
I20260927 18:42:07.723560 28922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.723793 28922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17b6d21913a74343984bf9e939b3d547, State: Initialized, Role: FOLLOWER
I20260927 18:42:07.724114 28847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: Bootstrap starting.
I20260927 18:42:07.724143 28922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.724730 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.724998 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.725502 28922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:07.726867 28847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: Neither blocks nor log segments found. Creating new log.
I20260927 18:42:07.727768 28847 log.cc:824] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: Log is configured to *not* fsync() on all Append() calls
I20260927 18:42:07.729941 28847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: No bootstrap required, opened a new log
W20260927 18:42:07.730715 29029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:42:07.730798 29029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:42:07.728886 28854 master_runner.cc:186] Error getting master registration for 127.6.4.62:43185: OK, instance_id { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" instance_seqno: 1790534527247431 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:42:07.732656 28854 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:42:07.732627 28847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.732839 28847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 290b182021f44713bf2daaf5bedd1e43, State: Initialized, Role: FOLLOWER
I20260927 18:42:07.733131 28847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.733547 29032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.733671 29032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.734331 28847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:42:07.734448 28821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 18:42:07.734558 28821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:07.735754 28821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.736742 28821 leader_election.cc:288] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [CANDIDATE]: Term 1 election: Requested vote from peers 17b6d21913a74343984bf9e939b3d547 (127.6.4.61:46631), 3c36c09af0ab40e28c7837b6626c8014 (127.6.4.60:36481)
I20260927 18:42:07.737288 28896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "290b182021f44713bf2daaf5bedd1e43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "17b6d21913a74343984bf9e939b3d547"
I20260927 18:42:07.737475 28971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "290b182021f44713bf2daaf5bedd1e43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3c36c09af0ab40e28c7837b6626c8014"
I20260927 18:42:07.737546 28896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:07.737720 28971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:42:07.738803 28896 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 290b182021f44713bf2daaf5bedd1e43 in term 1.
I20260927 18:42:07.738992 28971 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 290b182021f44713bf2daaf5bedd1e43 in term 1.
I20260927 18:42:07.739476 28798 leader_election.cc:302] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 290b182021f44713bf2daaf5bedd1e43, 3c36c09af0ab40e28c7837b6626c8014; no voters:
I20260927 18:42:07.739755 29032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:42:07.739945 29032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [term 1 LEADER]: Becoming Leader. State: Replica: 290b182021f44713bf2daaf5bedd1e43, State: Running, Role: LEADER
I20260927 18:42:07.740284 29032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } }
I20260927 18:42:07.742053 29032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 290b182021f44713bf2daaf5bedd1e43. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.742144 29032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: This master's current role is: LEADER
W20260927 18:42:07.743010 29043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:42:07.743085 29043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 18:42:07.744532 28800 tablet.cc:2369] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:42:07.745396 28779 master_runner.cc:263] No leader master found from master 290b182021f44713bf2daaf5bedd1e43
I20260927 18:42:07.745463 28779 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:42:07.745913 29051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:42:07.748337 29051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:42:07.756453 28971 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [term 1 FOLLOWER]: Refusing update from remote peer 290b182021f44713bf2daaf5bedd1e43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:07.756531 28896 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [term 1 FOLLOWER]: Refusing update from remote peer 290b182021f44713bf2daaf5bedd1e43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:42:07.757206 29032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:07.757644 29032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:42:07.761890 29003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 290b182021f44713bf2daaf5bedd1e43. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.762059 29003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.762739 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 290b182021f44713bf2daaf5bedd1e43. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.762953 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.772212 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.744491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.6.4.62:43185)
0927 18:42:07.744597 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.744602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.752434 (+ 7832us) server_negotiation.cc:407] Connection header received
0927 18:42:07.753885 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.753888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.753936 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.753984 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.757421 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.758076 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.761591 (+ 3515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.761943 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.762037 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.766247 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.766253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.766256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.766292 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.767869 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.770446 (+ 2577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.770450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.770451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.770540 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.770814 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.770817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.770819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.771051 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.771159 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.771887 (+ 728us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.772019 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:42:07.774861 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.745374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.6.4.61:46631)
0927 18:42:07.745483 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.745486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.752600 (+ 7114us) server_negotiation.cc:407] Connection header received
0927 18:42:07.752779 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.752782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.752844 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.752893 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.754851 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.755508 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.761052 (+ 5544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.761381 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.761472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.768409 (+ 6937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.768413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.768416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.768449 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.769966 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.771457 (+ 1491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.771461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.771463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.771548 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.773634 (+ 2086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.773638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.773640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.773899 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.774009 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.774570 (+ 561us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.774689 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:42:07.777721 29048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.777881 29048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:07.777997 29055 mvcc.cc:205] Tried to move back new op lower bound from 7334029425681883136 to 7334029425627607040. Current Snapshot: MvccSnapshot[applied={T|T < 7334029425681883136}]
I20260927 18:42:07.779393 29003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.779525 29003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.779944 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:07.745101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.6.4.60:36481)
0927 18:42:07.745208 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.745212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.752437 (+ 7225us) server_negotiation.cc:407] Connection header received
0927 18:42:07.752579 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.752583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.752641 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.752733 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.753702 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.754560 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.757193 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:07.757565 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:07.757640 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:07.776337 (+ 18697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:42:07.776341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:42:07.776344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:42:07.776375 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:42:07.778070 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.778644 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.778647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.778649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.778735 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:42:07.779023 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:42:07.779026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:42:07.779028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:42:07.779228 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:42:07.779336 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:07.779662 (+ 326us) server_negotiation.cc:299] Negotiation successful
0927 18:42:07.779782 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:42:07.781831 29057 mvcc.cc:205] Tried to move back new op lower bound from 7334029425681883136 to 7334029425627607040. Current Snapshot: MvccSnapshot[applied={T|T < 7334029425681883136}]
I20260927 18:42:07.781991 29032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.782116 29032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43 [sys.catalog]: This master's current role is: LEADER
I20260927 18:42:07.783361 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "290b182021f44713bf2daaf5bedd1e43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "290b182021f44713bf2daaf5bedd1e43" member_type: VOTER last_known_addr { host: "127.6.4.62" port: 43185 } } peers { permanent_uuid: "17b6d21913a74343984bf9e939b3d547" member_type: VOTER last_known_addr { host: "127.6.4.61" port: 46631 } } peers { permanent_uuid: "3c36c09af0ab40e28c7837b6626c8014" member_type: VOTER last_known_addr { host: "127.6.4.60" port: 36481 } } }
I20260927 18:42:07.783486 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:42:07.784710 29051 catalog_manager.cc:1347] Generated new cluster ID: 9d2e7b10e5ef43f99f5c090448bffb41
I20260927 18:42:07.784775 29051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:42:07.798341 29056 mvcc.cc:205] Tried to move back new op lower bound from 7334029425681883136 to 7334029425627607040. Current Snapshot: MvccSnapshot[applied={T|T < 7334029425681883136}]
I20260927 18:42:07.852219 29051 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:42:07.853561 29051 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:42:07.871546 29051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 290b182021f44713bf2daaf5bedd1e43: Generated new TSK 0
I20260927 18:42:07.872454 29051 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:42:07.914497 29016 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.911569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.6.4.60:36481)
0927 18:42:07.911694 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.911698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.913383 (+ 1685us) server_negotiation.cc:407] Connection header received
0927 18:42:07.913608 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.913612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.913663 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.913713 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.914360 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36412: BlockingRecv error: recv got EOF from 127.0.0.1:36412 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260927 18:42:07.915829 28851 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.912388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.6.4.62:43185)
0927 18:42:07.912495 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.912499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.914989 (+ 2490us) server_negotiation.cc:407] Connection header received
0927 18:42:07.915143 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.915146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.915194 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.915240 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.915704 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33836: BlockingRecv error: recv got EOF from 127.0.0.1:33836 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:42:07.917272 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.913061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.6.4.61:46631)
0927 18:42:07.913194 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.913198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.916336 (+ 3138us) server_negotiation.cc:407] Connection header received
0927 18:42:07.916564 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.916567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.916618 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.916666 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.917145 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37302: BlockingRecv error: recv got EOF from 127.0.0.1:37302 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260927 18:42:07.920984 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.919481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.6.4.61:46631)
0927 18:42:07.919572 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.919576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.920002 (+ 426us) server_negotiation.cc:407] Connection header received
0927 18:42:07.920251 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.920255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.920330 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.920381 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.920886 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37310: BlockingRecv error: recv got EOF from 127.0.0.1:37310 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260927 18:42:07.925463 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.924622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.6.4.61:46631)
0927 18:42:07.924726 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.924730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.924744 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:42:07.924869 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.924872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.924915 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.924975 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.925355 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37316: BlockingRecv error: recv got EOF from 127.0.0.1:37316 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:42:07.930269 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.929222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37322 (local address 127.6.4.61:46631)
0927 18:42:07.929369 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.929373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.929504 (+ 131us) server_negotiation.cc:407] Connection header received
0927 18:42:07.929657 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.929661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.929706 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.929767 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.930152 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37322: BlockingRecv error: recv got EOF from 127.0.0.1:37322 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260927 18:42:07.937109 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.935937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.6.4.61:46631)
0927 18:42:07.936144 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.936150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.936339 (+ 189us) server_negotiation.cc:407] Connection header received
0927 18:42:07.936462 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.936465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.936512 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.936576 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.936997 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37338: BlockingRecv error: recv got EOF from 127.0.0.1:37338 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260927 18:42:07.948616 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.947303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37350 (local address 127.6.4.61:46631)
0927 18:42:07.947431 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.947436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.947711 (+ 275us) server_negotiation.cc:407] Connection header received
0927 18:42:07.947832 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.947835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.947899 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.947996 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.948481 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37350: BlockingRecv error: recv got EOF from 127.0.0.1:37350 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260927 18:42:07.967609 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:07.966471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37366 (local address 127.6.4.61:46631)
0927 18:42:07.966594 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:07.966599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:07.966824 (+ 225us) server_negotiation.cc:407] Connection header received
0927 18:42:07.966926 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:07.966929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:07.966979 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:07.967043 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:07.967490 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37366: BlockingRecv error: recv got EOF from 127.0.0.1:37366 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:42:08.003121 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:08.002014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.6.4.61:46631)
0927 18:42:08.002172 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.002177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.002280 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:42:08.002430 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.002434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.002492 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.002561 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:08.003013 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37376: BlockingRecv error: recv got EOF from 127.0.0.1:37376 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260927 18:42:08.070382 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:08.069288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.6.4.61:46631)
0927 18:42:08.069430 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.069434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.069571 (+ 137us) server_negotiation.cc:407] Connection header received
0927 18:42:08.069706 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.069709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.069755 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.069823 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:08.070265 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37390: BlockingRecv error: recv got EOF from 127.0.0.1:37390 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260927 18:42:08.201563 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:08.200507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.6.4.61:46631)
0927 18:42:08.200661 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.200665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.200864 (+ 199us) server_negotiation.cc:407] Connection header received
0927 18:42:08.200925 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.200927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.200978 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.201039 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:08.201445 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37396: BlockingRecv error: recv got EOF from 127.0.0.1:37396 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260927 18:42:08.461081 28927 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:42:08.459962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.6.4.61:46631)
0927 18:42:08.460131 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.460137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.460279 (+ 142us) server_negotiation.cc:407] Connection header received
0927 18:42:08.460434 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.460437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.460484 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.460555 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:42:08.460971 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37404: BlockingRecv error: recv got EOF from 127.0.0.1:37404 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 18:42:08.710896 29014 catalog_manager.cc:1259] Loaded cluster ID: 9d2e7b10e5ef43f99f5c090448bffb41
I20260927 18:42:08.711000 29014 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: loading cluster ID for follower catalog manager: success
I20260927 18:42:08.712672 29014 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: acquiring CA information for follower catalog manager: success
I20260927 18:42:08.713832 29014 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c36c09af0ab40e28c7837b6626c8014: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:42:08.732075 29029 catalog_manager.cc:1259] Loaded cluster ID: 9d2e7b10e5ef43f99f5c090448bffb41
I20260927 18:42:08.732170 29029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: loading cluster ID for follower catalog manager: success
I20260927 18:42:08.733840 29029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: acquiring CA information for follower catalog manager: success
I20260927 18:42:08.735153 29029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17b6d21913a74343984bf9e939b3d547: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:42:08.978602 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:08.975073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.6.4.61:46631)
0927 18:42:08.975325 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.975329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.975429 (+ 100us) server_negotiation.cc:407] Connection header received
0927 18:42:08.975564 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.975567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.975694 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.975788 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:08.975992 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.976551 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.977384 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.977600 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.977664 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:08.977887 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:08.978308 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:08.978393 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:08.978419 (+ 26us) server_negotiation.cc:299] Negotiation successful
0927 18:42:08.978476 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
I20260927 18:42:08.984687 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:08.981431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.6.4.60:36481)
0927 18:42:08.981632 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.981635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.981648 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:42:08.981723 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.981725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.981843 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.981935 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:08.982099 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.982587 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.983358 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.983565 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.983631 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:08.983865 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:08.984341 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:08.984427 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:08.984472 (+ 45us) server_negotiation.cc:299] Negotiation successful
0927 18:42:08.984523 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":58,"threads_started":1}
I20260927 18:42:08.991125 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:08.987453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.6.4.61:46631)
0927 18:42:08.987636 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.987641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.987912 (+ 271us) server_negotiation.cc:407] Connection header received
0927 18:42:08.988073 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.988077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.988154 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.988213 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:08.988421 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.989003 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.990052 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.990372 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.990456 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:08.990659 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:08.990730 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:08.990789 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:08.990928 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:42:08.990986 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 18:42:08.991487 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:08.987491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.6.4.60:36481)
0927 18:42:08.987597 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.987600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.988005 (+ 405us) server_negotiation.cc:407] Connection header received
0927 18:42:08.988150 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.988153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.988200 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.988261 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:08.988594 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.989353 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.990360 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.990684 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.990779 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:08.991057 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:08.991131 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:08.991174 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:08.991319 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 18:42:08.991369 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:42:08.992159 29140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:42:08.987521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.6.4.62:43185)
0927 18:42:08.987782 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0927 18:42:08.987787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:42:08.988023 (+ 236us) server_negotiation.cc:407] Connection header received
0927 18:42:08.988150 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:42:08.988153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:42:08.988310 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:42:08.988419 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:42:08.988664 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.989464 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.990400 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:42:08.990721 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:42:08.990798 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:42:08.991245 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:42:08.991600 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:42:08.991687 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:42:08.991930 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 18:42:08.991991 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":61,"threads_started":1}
I20260927 18:42:08.999838 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28779
I20260927 18:42:09.005384 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28854
I20260927 18:42:09.010619 6160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFEZSEE/build/debug/bin/kudu with pid 28928
2026-09-27T18:42:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3002 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5990 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271463 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST